Some time back, I was thinking about the problem of converting natural language to a relational query language, e.g. SQL.
Language models nowadays are not robust estimators, and they struggle to generate complex constructions in formal languages. They're not even fit for performing their own uncertainty quantification either. I think this in part has to do with the choice of parametric forms in language models and the optimization programs used to train them. But, I won't get into that here.
Anyways, how do you convert a natural language query to a relational query from the "noisy" outputs of a language model?
I'd first pose this as a statistical inverse problem: you're basically estimating a partial function from natural language (NL) to relational query language (SQL, in this case). I'd consider it partial since not all natural language sentences correspond to relational queries. The inverse problem is converting NL to SQL. The forward problem is converting SQL to NL.
Considering the limitations of present language models, I think the forward problem more well-posed than the inverse if you do it in a bottom-up manner. And, that's totally feasible because SQL is inductively defined by a grammar.
Imagine that you have two relations and you know the natural language analogue for both of them. Now, you're asked "what's the natural language analogue of the equijoin of those two relations"? That's way easier for today's language models to solve because it leverages these "language models" native language: natural language!
But, what's the use of solving this "forward problem" if we want to actually solve the inverse problem? Well, given some noisy results of the inverse (e.g. a language model's NL-to-SQL conversions), you can use the less noisy forward results of those outputs, and then measure pairwise consistency/similarity.
This can be followed to a solution that resembles how the Bayesian line of statistics exploits data generating process (DGP). Altogether, this basically reduces the difficulty of NL-to-SQL to semantic similarity over natural language (consistency checking step), which is very easy nowadays for natural language models.
You could make this process more rigorous with intermediate DSLs/IRs that have to do with your relational schema just as compilers do.
This whole idea can be followed in some interesting directions, like "generating meaningful views".