1. Exposure to rich, authentic language

The texts – or the language – that a computer can understand or generate depend on its capacity for NLP. Computer scientists added the adjective ‘natural’ because the parsing of programming language(s) was possible, and necessary, before they turned to parsing texts produced by humans. In early NLP, computational linguists wrote grammatical rules and compatible dictionaries in programming languages such as Prolog and LISP. Rules and items were written by hand, relying on different (mathematical) grammar formalisms. This made the development process slow, error-prone, computationally expensive, and labor-intensive. This might be the main reason why the coverage and robustness of ICALL systems and applications remained limited over the years. Parsing a single sentence – the analysis of the grammatical constructions and the production of an equivalent information structure, something similar to a syntactic tree, which the computer could “understand” – took from a couple of seconds to a few minutes, depending on the computer hardware and the efficiency of the parsing algorithm. This approach to NLP is called symbolic, because it uses and processes symbols for syntactic phrases, such as NP for a noun phrase and VP for a verb phrase, and for lexical items, such as N for a noun and V for a verb, and their grammatical feature structures. Symbolic NLP in ICALL resulted in sentence-based language learning activities in a tutorial system. Since the dictionary was also hand-written and hence usually small, the language to which students using the ICALL system were exposed was limited to the vocabulary of a textbook at best.

The part #0 gave a general introduction. Here we have the first section on what language learners should expect of generative-AI tools. Other parts will follow.

My inspiration for this title came from the book  
Snyder, T. (2017). On tyranny: Twenty lessons from the twentieth century. Tim Duggan Books. 

I am sharing these early drafts of a book chapter I published in 
Yijen Wang, Antonie Alm, & Gilbert Dizon (Eds.) (2025), 
Insights into AI and language teaching and learning. Castledown Publishers. 

https://doi.org/10.29140/9781763711600-02.

In the 1990s, more electronic corpora (large, principled collections of texts) also in languages other than English became available. The approach to NLP that relies on the mathematical analysis of large corpora has been called statistical NLP. In this approach, language patterns are detected in corpus analyses. For these patterns or contiguous sequences of words – called n-grams with n being the number of words in each and every sequence – the probability of one word following the other(s) is calculated. In their simplest form, the probabilistic connection of linear word sequences is calculated. This results in a wider coverage of language, because of the underlying use of large corpora. However, the limitation was that, for example, long-distance dependencies as in the following sentence still posed a problem as they had in symbolic NLP.

The student who had finally given the right answer proceeded to ask the next question.

For any human reader, it is immediately clear that it is ‘the student’ who ‘proceeded’ and not ‘the right answer’. For the computer, this connection between grammatical subject and finite verb poses a challenge because the words are not in the same n-gram(s) and the pattern cannot be detected easily.

This and other challenges were overcome by relying on artificial neural networks (ANNs), models that are inspired by the neural networks of human brains (for a comprehensive overview of how GPTs (generalized pre-trained transformers) work, see Wolfram, 2023, February 14). ANNs are multidimensional and do not only rely on linear sequences of words. Their individual nodes, the neurons, receive input and send output to other neurons. The processing of input to produce output is basically done through a mathematical equation. Thus, this output depends on the (probabilistically) weighted input. The network learns by adjusting the weights, which multiply different input values and biases, the latter are added independently of the input, to improve the accuracy of the result. If this machine learning relies on neurons organized in multiple layers – the input layer, the output layer, and in-between two or more hidden layers, then we talk about a deep network and deep learning (LeCun et al., 2015). “GPT-3 has 96 layers. GPT-4’s exact number of layers hasn’t been publicly disclosed, but it is expected to be significantly larger than GPT-3” (Microsoft Copilot, 2024, December 26). Deep learning and ANNs are the underpinnings of the large language models (LLMs) (for an accessible overview of ANNs and LLMs, see Naveed et al., 2024), which in turn are the backbone of GenAI chatbots such as ChatGPT (OpenAI), Claude (Anthropic), Copilot (Microsoft), and Gemini (Google). Thus, LLMs essentially rely on enormous corpora of texts scraped from the internet and on machine-learned neural networks. In these ANNs, individual tokens – which can be individual letters, words, and parts of a word – are represented by long lists of numbers, which are called word vectors. The parameters in the network, which are tiny little rules and steps, help to determine which word follows the previous word. “GPT-3 has 175 billion parameters, which include the weights and biases of the neurons. GPT-4 is speculated to have trillions of parameters, though the exact number hasn’t been confirmed” (Microsoft Copilot, 2024, December 26).

This new computational approach is far removed from the reliance on linguistic rules and items in early NLP and ICALL, because it is steeped in the complex calculations in the hidden layers of the LLM and arrays upon arrays of numbers. That’s why GenAI’s coverage, scope, and speed of NLP is vastly superior to previous systems in ICALL. Therefore, we can argue that students using GenAI are exposed to rich language at the paragraph and not only the sentence level. But is this generated language authentic? In an early paper on authenticity in the language classroom, Breen (1985) proposes that “that authentic texts for language learning are any sources of data which will serve as a means to help the learner to develop an authentic interpretation” (p. 68). The question then becomes: can a learner develop an authentic interpretation of a turn or text generated by a GenAI chatbot or a translation rendered by a GenAI machine translation tool? Since the generated texts are certainly well-formed and plausible, they appear to provide a good basis for the learner’s interpretation and thus for language learning. Also, because they are based on actual language use as found in the texts on the internet, which were used to train the LLM, we have another indication that generated texts in chat with a GenAI or a translation from a GenAI potentially qualify as authentic. However, the real key to authenticity of language is found in communication.

References

Breen, M. P. (1985). Authenticity in the Language Classroom. Applied Linguistics, 6(1), 60-70. https://doi.org/10.1093/applin/6.1.60

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep Learning. Nature, 521, 436–444.

Microsoft Copilot. (2024, December 26). How many nodes and layers does the ANN of the GPT large language model have? Microsoft Copilot.

Naveed, H., Khan, A. U., Qiub, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2024). A Comprehensive Overview of Large Language Models. https://dx.doi.org/10.48550/arxiv.2307.06435 http://arxiv.org/pdf/2307.06435

Wolfram, S. (2023, February 14). What is ChatGPT doing … and why does it work? Stephen Wolfram Writings. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work

… to be continued …