Template macros too obscure?

Hi,

I have digged into MPS for a couple of days now and I am excited and stunned at the same time. It is amazing how the whole system bootstraps itself with its core concepts. The language design and editor construction is just amazing. However, I have some critique concerning the generator template language.

The anonymous macros hide their logic and do not help the reader with some descriptive name like functions in programming languages would. The template reads "dollar this, dollar that..." and the background semantics of the macros is hidden and can only be retrieved with the inspector. The template DSL is not descriptive and it is impossible to understand the template without the inspector.

I find it extremely difficult if not impossible to create a mental model of a given generator, which I have not designed myself. It is too hard to keep all the semantics of the many anonymous macros in my head. The logic of a generator is scattered across the mapping spec, the external templates, and the hidden macros.

Also, i find the numerous [null] arguments to be an indicator that this concept is somewhat skewed.

Was there a specific reason why an anonymous macro concept has been chosen in favour of an ordinary functional decomposition scheme like in Open Architecture Ware, for instance?

If I missed some detail, please let me know.

Cheers

Hermann

8 comments
Comment actions Permalink

Hi, Hermann


I love anonymous macro because I don’t have to think of dozens (or hundreds) of unique descriptive names while writing generator:)
In MPS you create a macro and immediately define its semantic in inspector (which is not necessarily a single function BTW – see MAP_SRC macro for the instance http://www.jetbrains.net/confluence/display/MPS/Generator+Language ).

In oAW to do the same (I don't have experience with oAW though) you will have to create macro, then think of descriptive function name, then leave your template and go somewhere to declare your function, ensure that the name is unique (will need another descriptive name at times), then return to template and add the function call.
Whenever you change your template you will have to synchronize those changes with functions declaring somewhere else.

In addition, return type of anonymous macro functions in MPS can vary depending on the place where the macro is used. This is hardly achievable with externally defined functions.

Regards, Igor.

0
Comment actions Permalink

Hi Igor,

I understand that writing anonymous macros is fun because they let you get things done more quickly and you don't have to be inventive with names. But isn't it a good practice in programming to show the intent of a program by choosing meaningful names? We do this all the time while programming. I hope you do it too ;-)

The MAP-SRC macro for instance can replace the input node. Silently. Nobody can see unless using the inspector. I mean, relacing the input node is something the template reader should really be aware about instantly when studying the template. The template source code is one logic fragment, which alone cannot express (at least give hints) what the generator is really doing.

To put things to extremes: Imagine a programming language where all functions are called f() and you need an inspector to look into them. You would hate it.

oAW behaves like ordinary programming languages here and rests on functional decomposition. Which of course has another benefit: reuse. How many macros have you designed, which do exactly the same, but are still separately maintained? I think I found quite a few while browsing through the languages. That also reduces the number of names to be invented. I would consider function names as a part of a domain-specific template language.

I find the idea to have a set of standard macros with a pre-defined logic perfectly ok, because those are the functions, which are very common in a generator. In languages like Ruby for example, we would probably pass closures as arguments if it is just a little bit of logic, or factor out into functions if it is more complicated. In any case, it is visible for the reader immediately. Either the code or some descriptive function name.

I'm not experienced enough to have an opinion about the context-dependent signature of macros. It sounds like a little bit of magic, though.

You see, I was working my way through the language and editor design and found beautiful abstractions and concepts. And then I hit the macros. I thought "come on guys, you have been so inventive up to here, are macros all you've got?" Don't take this personal, please. I like to give honest feedback to you, because this amazing product deserves it. Otherwise I really wouldn't care.

Regards

Hermann

0
Comment actions Permalink

Hi Hermann,

In MPS we do reuse macro code when necessary. Generator of almost every language contains model like 'util' or 'helper' with collection of reusable methods which are called from inside macros.

You are holding the opinion that descriptive mane or macro code being shown in template code would make the template more readable. Sounds reasonable, but on the other hand, this would obscure the template code with often (how often?) unnecessary details.

oAW xpand, as I see it, is one-step model-to-text generation framework.
MPS generator framework goes far beyond that because it supports multi-step model-to-model transformation, with gradual lowering of level of abstraction until it reaches level of target platform (java for instance). This way we achieve language composability.
We’ve gained huge experience with generators and our generators are really awesomely complex at times :-)
At this moment, in my instance of MPS, I can count 8861 macro in about 100 languages.
And yet, none of developers seem unhappy with macro anonymity (until now maybe? :-)).
This tells me that our approach is not as bad in practice as it might seem in theory.

Anyway, we will continue to collect feedback. Probably, what you are talking about is worth to try.

Unfortunately, I know little about a real life applications of oAW. Does anybody develop oAW generators on that scale?

Regards, Igor.



Attachment(s):
8861-macro.PNG
0
Comment actions Permalink

Igor,

we do agree that the template code should be on a rather abstract level and not be cluttered with silly little details.

I suspect that you folks are a bit more clever than the average, that's why you are doing alright with the abstract macros B-) But seriously, I thought "hey, they built this huge thing and it does work", so it can't be that bad. However, there is probably room for improvement for a lower entry barrier for average mortals...

I browsed the base language templates and the attached examples about says it all why I consider it very difficult to read. A big anonymous blob of nested macros, which gives me not the slightest hint what the actual purpose is, unless I inspect (and keep in mind) each and every macro.

Looking at it, an optional one-line description for each macro, which could be displayed either by bubble help or between the $ $ separators in a different font or color could do the job.

This would read like

$IF some textual comment here$ [ $COPY_SRC i copy fancy stuff$ [ ....

The one-liner would fit nicely in the macro definition in the inspector. It wouldn't break the macro concept and give very valuable help to the reader.

I have to maintain a very large oAW-based generator currently. oAW has two languages: "Expand" is the template language, whose task is to produce text. "Extend" is the functional language for model-to-model transformation, which can also be used in the Expand templates to hide details. Extend is roughly equivalent to the macros or base language scripts in MPS. A well-written Expand template is quite readable and has a flat learning curve. Extend can also call Java methods, if things get too complicated for this comparatively simple language. In the worst case (like mine), a generator's logic is scattered accross three totally different languages (Expand, Extend, Java). There is also room for improvement :-]]

Ciao

Hermann



Attachment(s):
mps_macro.jpg
0
Comment actions Permalink

Hi Hermann,

I've added optional comments to macros - shown on the lettuce-green background in template - check out attachement.

Regards, Igor.



Attachment(s):
macro_with_comment.PNG
0
Comment actions Permalink

Igor,

WOW, you are quick! Now I am actually able to understand the template without the inspector. It is so much helpful to see the IF conditions in a glance. With good comments the template almost reads like a story. Literate programming, here we come :-D. I am sure that this will safe a lot of time in generator maintenance.

Good stuff, I hope you keep it.

Cheers

Hermann

0
Comment actions Permalink

Yes, it looks like the comments are helpful :-).

Surprisingly, they've highlighted few places where this template could be optimised :

- $IF$has result type [$COPY_SRC$result type[...]]  - can be implemented as only COPY_SRC  (if COPY_SRC terurns null 'mapped node', then generator will skip it)

- all $LOOP$ [$COPY_SRC$ [...]] here can be equally replaced with $COPY_SRCL$ macro


Thank you for the feedback!

0
Comment actions Permalink

Not that I like to sound too clever here, but it does not surprise me that once the intention of the macros is revealed, suddenly the perception of the template becomes more complete.

As a matter of fact, I had some more thoughts on symbols (names) in classic programming languages. Symbols are required by the language technology (compilers, interpreters) to establish a relation between things (for instance between a function definition and a function call). So, we are asked to give them meaningful names to understand the relation ourselves, or to build a vocabulary, respectively. The compiler doesn't care which names we choose, it is exclusively for the human reader that we have this convention.

In MPS however, since the editors build the structure directly and there is no parsing, references can be established without symbols. The tool helps to find the referenced things and keeps the physical relation hidden. This opens possibilities of representing a reference in various ways. As for the anonymous macros, showing the textual comment is a good choice and it beats classic symbolic, language-conformant names by far!

I am not saying that symbols become superflous altogether, since we are used to give names to things, but I find it worth contemplating why some kind of symbols are necessary in languages and whether they still are in a technology like MPS.

A fascinating new "ecosystem", really.

0

Please sign in to leave a comment.