So, I've had some thoughts about MPS scopes and references for a while now and I wanted to get some input. Personally, I feel that the current way references and scopes are done is very limited. A g...

This is an auto-generated question from the MPS Slack community: So, I've had some thoughts about MPS scopes and references for a while now and I wanted to get some input. Personally, I feel that the current way references and scopes are done is very limited. A good example of this limitation is the concept functions extension. The extension provides a way to use defined concepts as parameters to custom functions embedded in languages. Normally in baselang, local variables, method parameters, and some other stuff, can all be a referenced by the VariableReference, but ConceptFunctionParameters cannot be. That is obvious as concept declarations don’t extend the necessary concept and can’t, but this causes a lot of problems. Because of this, the mechanism for scopes can not address all “variables” that are usable, if a node wants to filter all the variables in a scope, it has to consider both regular variables and concept parameters as separate cases. Another case when I see limitations of this system appeared during the implementation of one of my teams DSLs within MPS. The language has the concept of a struct similar to C and there are other language concepts that create virtual structs. These virtual structs are never defined directly by the user (instead transitively through the existence of other language constructs) but can still be referenced in the language just like any other struct in terms of types and field dereferencing. This is problematic as references only deal with nodes that exist and that problem also transfers to the scopes as they can not filter a struct that does not exist. A solution to this issue I think presents itself with a new type of “reference” and way of scopes. One where nodes themselves can be treated as complex references and scopes don’t provide a list of nodes that can be referenced, rather a list of ways to construct nodes to serve as these complex references. As a solution to the first problem mentioned, the concept VariableReference would no longer store a reference to the variable declaration node, instead it would store a node that stores that reference and could also store a different node that references the concept parameter declaration. VariableReference would in essence have a child that is some abstract concept that serves to represent the reference of a variable defined somewhere. This also works for the second example as it allows arbitrary nodes to represent a reference to some virtual construct that doesn’t exist as just a node, but rather can be concretely described via properties and references (potentially multiple) to other nodes. The point is to allow scopes to control what is usable in code more abstractly, allowing scopes to describe what variables can be used without limiting it to direct references of nodes of a particular concept. And I recognize that this is already sort of feasible, but from what I see there are still a few roadblocks to making something like this usable. Reference resolve info would need to be extended to existence of these complex reference nodes and nodes themselves would potentially change instances if a reference became unavailable. I don’t know how these references would work with generators as you would want these references processed during the reference stage of a gen step, but they are nodes and not references. Smart references might need to have some extension to nodes. There are also other mechanisms to potentially address these problems. Shadow Models and DclareForMPS both provided mechanisms for incrementally maintaining models in other forms. These plugins could maintain driven nodes that represent the model in the form that can be appropriately referenced. Such as Shadow models maintaining a separate traditional MethodParameter for each defined ConceptFunctionParameter such that they can be a reference in a VariableReference instance. I like shadow models for this more than Dclare as SM does not require the modification of the actually defined model, but having the generated method parameter exist under a different model than what serves to define it might introduce its own problems like with the editor action go to declaration as it would make more sense to go to the conceptfunctionparameter than the generated MethodParameter I think. Really what I want is to be able to describe these extensions and more complex language behaviors without having to define them in such a way that they are incompatible with the existing tooling, which I see very often in the extensions of baselang Thanks :slightly_smiling_face: