Can you abstract multiple steps within MPS itself?
I am a huge fan of the idea of meta-programming, and I've thought a lot about how to implement such a system myself, but have never gotten around to doing it (too busy with other work!) However, I had hoped that eventually someone else would try to build such a thing, and I'm glad you guys have taken a stab at this.
One general comment: perhaps I am in the minority here, but I find the UI conventions used in MPS highly non-intuitive. Perhaps these are Java standard UI conventions --- but I really think they ought to be rethought. For example, double-clicking on a selection in the Ctrl-Space code completion popup should select the item; instead, you have to click then hit enter. Or another example, when you're able to edit in a text edit field, there should be some sort of blinking bar cursor the way every other text edit field in the universe operates. It seems to me that following basic UI conventions common in, say, web browsers or pretty much every windowing system from Windows to KDE to Gnome to whatever would go a long way to improving the user experience of people unfamiliar with the tool.
That's a fairly minor quibble however. A much bigger complaint is that it takes far too many steps to do common tasks. Conceptually, I believe (particularly with meta-programming systems) in "simple things should be simple, complex things should be possible." The Hello World example takes a huge number of separate steps (clicking on things, creating things, etc., etc.) which should be boiled down to the absolute minimum number of steps. Which brings up the question: I would hope (but I haven't studied MPS carefully enough to know) that it might be possible to build an MPS language that is meta to the whole MPS process. I.e., would it be possible to solve the problem of a huge number of steps by writing an MPS language that would "macro-ize" some of these steps?
For example, the process of creating a class template that just maps a single text constant property in a concept to a System.out.println(property); code generation should be an absolutely straightforward task. It shouldn't take umpteen kazillion separate UI operations. I should be able to do something, conceptually, like:
1) define concept HelloWorld with single string constant property helloText
2) define template which says: concept HelloWorld maps to a class definition in baseLanguage, HelloWorld[possible macro to disambiguate]
3) define template method which says: static method main() with System.out.println(concept HelloWorld.helloText);
4) generate!
It seems to me the above 4 steps are conceptually all you should need and the UI ought to be able to boil these steps down somehow and do all the gobbledegook in the background to actually hook all this stuff up. Is it possible to do this already in MPS? I.e., can MPS operate in its own abstractions? I would imagine and hope the answer is yes, and I'm curious to find out the answer.
I also think that there should be some consideration for small mini-parsers to make it easier and more fluid to enter code. Structured editing is fine for a lot of things, but inputting System.out.println($HelloWorld) took far longer with all the mouse clicks and menu selections than it would have taken me to just type the above. Meta-programming should be faster than regular programming! There really should be more support for more free-form text editing and for many situations generating small parsers ought to be doable. Obviously there are all the problems of disambiguation with parsers, but it seems to me in those cases you just ask the programmer what they want and in 99.5% of the cases the parser ought to be able to figure out what the programmer means from context, at least for languages that are expressible in text format. Naturally this wouldn't work for all languages, but there ought to be a significant number of minor variations on baseLanguage that ought to allow this sort of thing.
Code completion and all that is great, but it ought not to be something that the programmer MUST use, it should be something that programmer CAN use.
Note: I've written IDEs myself, for my own use, and I found that at the level of things like instance variables, methods, classes, etc., having structured property editors is invariably faster than typing text for a wide variety of reasons. But for a few lines of imperative code --- it's often easier to just type text.
I suppose the above post is really three separate comments, but I welcome responses from the MPS team on any of these ideas.
Please sign in to leave a comment.
Sorry, re-reading it I realized I left out a couple of things. It's more like:
1) define language HelloWorld, and define concept HelloWorld with single string constant property helloText
2) define template which says: concept HelloWorld maps to a class definition in baseLanguage, HelloWorld[possible macro to disambiguate]
3) define template method which says: static method main() with System.out.println(concept HelloWorld.helloText);
4) create instance of concept HelloWorld, setting text property to "Hello, world"
5) generate!
I think most of these problems can be solved by giving stuctures default values.
If you create a new language, MPS can directly create a 'TopLevel' structure you only need to rename if you wish. Same goes for other things in the HelloWorld-example (file/path names can be derieved from project name etc.). With good defaults you can create HelloWorld with very few operation and still maintain flexibility. Its much easier to change/delete something you don't need sometimes then to always create it.
But I think they need to finish the query-language first, and then use it to create default expression.
About parsers: I don't like the idea.
Parsers are difficult to design and extend because of the ambuigities which uses to creep in the more the language grows. And they don't fit well into the syntax-editor concept, the user always have to think about what he's editing now.
I think, its possible to enhance the editor in a way that it surpasses editing speed of conventinal text editors - but its in a very early stage now and I suppose the primary reason for going EAP is to get new ideas about usabilitiy. So I expect a lot of improvements here - and I have to say that for the early state MPS shows a lot of promise.
Of course I realize the problem with parsers, but then again the whole point of programming systems is to make things easier for the programmer. With a structured editor it would not be necessary to resolve all ambiguities --- a single standard parser could resolve as many ambiguities as it could (which would often be all) and those it couldn't it would simply flag for resolution by the programmer. For the programmer to have to say "I am going to do a static member access now" and then "I am going to call an instance method now", etc., is laborious and is precisely the point of a programming system; i.e., to do the work that can be done automatically, automatically. If the ambiguity is unresolvable, then fine, fall back to asking the programmer for clarification, but please don't make me go System (access a static member) . out (call an instance method) . println ( (insert a string literal) ), every single damn time I want to write anything.
Would spreadsheets catch on if you had to write A1 (binary operator) + B1 (binary operator) * (function call) MAX( ... ) --- people would go bonkers doing that every time. For some things, small parsers are the way to go, and this system should have enough context information to do this for simple expressions in most cases without ambiguity.
The method-call-problem could be resolved with rewriting the source language to an intermediate language and resolving things like method calls. I think the baseLanguage is in the primary a target for the last step in code generation (before template expansion) and not realy useful as source language.
It's true, that the system should be as useful as possible for the programmer (or it won't be accepted), but in the moment I don't see any reason why there will be a need for a conventional parser.
But the usage will be a little bit different and the could take some time to get used to. For example you have to write something like '.' (to initiate Method call), 'Sy' (the system can complete automatically here if 'Sy' is non-ambiguous), , 'ou' (system autocompletes), , 'println', (system jumps to first parameter), <"> (system generates string-literal), "text")
In a conventional Editor like IDEA you have to type the same number of keys, only in a different order.
Same for operations: Its like UPN, you type: '+' A '*' B C
Also not more keypresses.
Also many conventional input patterns aren't necessary to type, because there will be a lot of language enhancement (thats what a system like MPS is for, right). So in the first example, you won't write this, but instead maybe "ctrl-W '"' text" which creates a write-expression (which is smarter and uses System.out.println or something else depending on context). This richness of semantic structures normaly comes together with a more and more complicated syntax to avoid abiguities, so a conventional language isn't easy expandable with new constructs like these. But with a structured editor it's no problem. And I think, that this advantage could be much greater than the slightly different input-style.
But even if you're right: Its much to early to give up now and use a conventional parser before they even have tried out a lot of things to make the editor as usable as possible and create a conventional parser instead. If in a year the usability is still to bad for most people to use MPS - THEN they should explore alternatives. But not earlier.
What you're describing is essentially the same thing I am talking about. In other words, rather than the programmer explicitly saying "I am about to write a method call", "I am about to insert a string literal", the editor/parser can infer this from context and from the characters the programmer is typing. Doing this is what I am calling "parsing" though you're imagining it happening as the programmer types, which is fine with me, although in some cases you will need to wait until the programmer has typed enough characters to disambiguate. In any event the basic mechanism is the same as what I was describing --- you need some sort of disambiguation algorithm to free the programmer from having to select everything in advance.
Making the programmer say "I am going to insert a string literal" is like forcing the programmer to write code in pre-parsed form, i.e., as though you were writing directly in parse tree format. I am saying we ought to make the tool as intelligent as possible and use the computer to disambiguate whenever it can, to avoid extra clicks and keystrokes. Writing code by selecting each type of operation feels like hand-coding parse trees, which really seems like a step backward.
More importantly, however, I'm still curious to know how reflexive MPS itself is. Can I write MPS languages that generate entire MPS configurations? As my first post indicates, I find the number of steps required to do a conceptually simple operation far more complex than is needed. Sure, you can avoid some of this via default values, but the whole point of meta-programming is being able to write languages that abstract out multiple complex steps like this. I'd like to see some reflexivity in the MPS configuration itself.
I fact you are editing the syntax tree directly (not the parse tree). That has obvious advantages but also disadvantages. I think the way to go with systems like MPS is to use more powerful languages instead of using a more powerful parsers. The editor could maybe use operator precedence to allow easy input of arithmetic expressions, but its more difficult compared to classic languages because these rules can change everywhere if you reuse language from the library.
I think you can resolve most of the problems you mentioned by creating multi-step conversions (some kind of language chaining). If someone wants a more expressive language he simply write a simple converter which translate certain constructs an underlying language. Language inheritance is another possibility, but I think its more dangerous, because it creates more possible wrong interaction between parts of the language.
The same goes for MPS itself. I hope that in the future, if you don't like i.e. the standard editor language, you can easiliy create your own 'wrapper' around the build in one and use this to edit editors in the way you want (same goes for query language etc.).
In principle MPS looks very reflexive but many things seem to be hardcoded in the moment because they need it to bootstrap the system somehow.
What I am recommending is NOT that MPS go completely to using parsers, but simply writing an optional editing mode which disambiguates semi-freeform text input on the fly. it could work in a simple way: if there is something that is ambiguous, highlight it in red or whatever, and then the programmer can choose from a selection list.
This eliminates the problem of parsers, because we're not expecting the parser to disambiguate everything, only what it can. The program is still stored in the system as a tree (with the exception of text that isn't fully disambiguated yet --- those could be stored as text and the node itself held in limbo until the text is disambiguated). This is a simple, uniform algorithm that would allow arbitrary extensions without any problems, it seems to me.
If people wanted to get fancy they could start to write specialized versions of the disambiguator that would allow for intelligent guessing and hints for parsing, the way real parsers work, but that would be an optional, not required, feature.
Anyway, can any MPS people comment on whether or not MPS will eventually be made reflexive, as a whole? If that happens then most of my objections can simply be dealt with in the LOP way: by writing new languages and editors.
Mitsu Hadeishi wrote:
I think for MPS to be successful in the long term, it will have to solve
this problem. It is widely known that strict-syntax editing does not
work, and in my opinion, MPS will have to closely approach text editing
speed/productivity to really succeed. However, at this time, I think
work in this area is probably not first priority.
I personally think this is a natural conclusion. What exists today in
IDEA as 'projects' and 'modules' is really just a coupld of
configuration languages. The project wizard and Paths settings are
really just specialized editors for the 'project language'. You could
already tell that part of it in MPS works that way, when you are
configuring certain parts of the language/model you are using MPS-like
editors. I imagine that the whole thing could be done as a set of languages.
Other things, like MPS itself could have special languages, such as to
automate MPS with 'macros' or something like that. The possibilities are
there in the long term.
--
Rob Harwood
Software Developer
JetBrains Inc.
http://www.jetbrains.com
"Develop with pleasure!"
>Other things, like MPS itself could have special languages,
>such as to automate MPS with 'macros' or something like
>that. The possibilities are there in the long term.
Yeah, that's what I was thinking. I'd love to work on such things once MPS has evolved to be able to support this. I sincerely hope this all becomes possible because it would make MPS a tremendously powerful, evolving self-referential system.