Governments and companies won’t stop the AI race

Researchers of artificial intelligence (AI) are calling for a pause on “giant AI experiments”, but it won’t happen, argues AI researcher Jürgen Schmidhuber.

In the past, I have signed letters warning about AI weapons. However, I don’t think the latest letterExternal link from leading researchers will have any significant impact because many AI researchers, companies and governments will ignore it completely.

The proposal frequently uses the word “we” and refers to “us”, the humans. However, there is no “we” that everyone can identify with. Ask ten people, and you will hear ten different opinions about what is “good”. Some of those opinions will be completely incompatible with each other. Don’t forget the enormous amount of conflict between the many people involved. 

Jürgen Schmidhuber 

Jürgen Schmidhuber is chief scientist at the Lugano, Switzerland-based company NNAISENSE, which aims at building the first practical general-purpose AI. He is a frequent keynote speaker on AI-related topics and advises various governments on AI strategies. Today’s most-cited neural networks used in applications such as Google’s speech recognition, Google Translate, Apple’s Siri and Amazon’s Alexa all build on work done in his labs.  

The letter also says, “If such a pause cannot be quickly enacted, governments should step in and institute a moratorium”. The problem is that different governments also have different opinions about what is good for them and for others. Great Power A will say, if we don’t do it, Great Power B will, perhaps secretly gaining an advantage over us. The same is true for Great Powers C, D and E. 

What role should Switzerland play here? It should of course try to stay at the forefront of AI research. Per capita, as of 2020, Switzerland was publishing far more papers at major AI conferencesExternal link than any other country. A moratorium would only hurt its competitiveness. 

A decades-long evolution

For example, ChatGPT is based on foundations developed by researchers who are now working at our Lugano-based company NNAISENSE whose motto is “AI∀” or “AI For All”.

ChatGPT uses an artificial neural network called Transformer, which relies on human attention mechanisms – meant to mimic cognitive attention – for natural language processing. Google employees published a now widely cited article on this in 2017 called “Attention is all you need”.External link I am happy about that because over 30 years ago I published a Transformer variantExternal link that is now called “Transformer with linearised self-attention” and recently presented more researchExternal link on the topic. 

GPT models also depend on “residual connections” – mechanisms that allow neural networks to become very deep and work much better. Residual connections are rooted in our LSTM of the 1990s (the most-cited artificial neural network of the 20th century) and our Highway Network (from May 2015), the first very deep feedforward neural network with hundreds of layers, whose principles can be found in thousands of applications today.

The present and future

What will happen next?  I have frequently compared ChatGPT and similar big language models to politicians. Politicians are usually good at talking, and they can give quick, ready-to-print answers to any question. They combine slogans they have used in many previous speeches in ever-new ways, so that no two answers are identical. Nevertheless, they often come up with platitudes without too much new insight or depth. 

Large language models like ChatGPT are not yet very good at thinking around many logical corners, as mathematicians and other scientists do. On the other hand, we have had neural networks for quite some time that can also learn in a similar manner, at least in principle. We now expect rapid progress here, which will make life easier for many people. 

Edited by Sabina Weiss and Veronica DeVore

The views expressed in this article are solely those of the author, and do not necessarily reflect the views of SWI swissinfo.ch.