Humans can interpret complex utterances they never heard before, like “Call the brother of the president of France”. This becomes possible because natural language is characterized by recursive hierarchical syntactic structures, which serve as input to compositional semantic interpretation. In our example, one can interpret the sentence following its syntactic structure: one can first identify the president of France, then his brother, then understand that the sentence urges us to call the latter.

Artificial neural networks, originally conceived as neurocognitive simulations, have proven to be practically useful computational models which define the state of the art in many linguistic applications such as machine translation, transcription, grammatical tagging etc. Part of the appeal of neural models comes from their ability to learn from data while being flexible enough to adapt to diverse tasks. These properties find clear analogs in human learning, including but not limited to the acquisition of linguistic knowledge.

Time is ripe to replicate the human ability to master understanding of any language by training artificial neural nets. Yet, compositionality and natural language structure remain far from trivial learning targets, and it is often beneficial to look at simplified formal languages to gain insights into abilities and limitations of different kinds of neural network models. The talk will discuss the results of ongoing work in this area, highlighting both challenges and promising directions.