What's the "output" of MPS?

Hello, Ran,

The output of MPS are generated files. They may be java, xml or any other kind of files. We have good java integration so in case of java files, MPS can compile them automatically.

Regards.

Konstantin

No, sorry, still don't get it.

What do I do with these "output" files? Let's say I choose a java files output - how can I then use these files in my java-based application?

Hello, Ran,

You can create a module/project in your favorite IDEs and set output folder of MPS as a source folder there. Then, you will be able to use these files in your project/module.

Regards,

Konstantin

You seem to be missing my point (or me your answers).

I don't (currently) care about the technical side of using the MPS output files. I want to understand how are these files can be used in my project. Why shouldn't I model my DSL in Java directly - what's the added value of MPS? And saying I do use the generated DSL in my project - how do I tie in the DSL to my existing API?

Ran,

MPS allows you to use domain specific notation inside of Java or in a standalone DSL which will rise level of abstract of code. In addition to ability to define a DSL, MPS provides a lot of infrastructure: editor, navigation, find usages, type system. You don't have to implement all these stuff from scratch.

You can tie your DSL to your API with generator. Generator will create classes that use your APIs. Actually, many APIs look like language expressed with classes and methods but because of limitations of them, code might look verbose. You can improve this with DSLs.

Regards,

Konstantin

I must be thick.

Is there an example in the MPS samples folder for such tie-ins?

If I understand correctly, I should look for:

1. "Client" code using the MPS generated classes.

2. "Server" code responding to MPS calls.

Ok - I'll try a different approach.

Let's say I've developed a genius langauge for querying data sources - GQL (genius query language).

Example of a query:

"select ds1.attr1, ds2.attr2 from ds1, ds2;"

Let's say I've modeled the language, created generators, editors and what's not (I'll get back to this in a bit), generated the java classes, packaged them into a jar and shipped to my client.

Querstion 1:

How would the client (java) use it?

1. languageProcessor.process("select ds1.attr1, ds2.attr2 from ds1, ds2;"); ?

2. Select.withDataSource("ds1").andAttribute("attr1").andDatasource("ds2").andAttribute("attr2").from("ds1", "ds2");

3. Some other way?

Question 2:

Now - what would happen when the client does whatever he needs to do in Q1? What part would he talk to (generator? actual language structure?)

Question 3:

How do I hook my genius backend processor to the langauge frontend?

Question 4:

Am I in the wrong direction completely?

Ran,

Is there an example in the MPS samples folder for such tie-ins?

There're probably no examples of such folder in MPS distribution, but if you download MPS source zip and open it in IDEA, you will see large number of source_gen directories.

If I understand correctly, I should look for:

1. "Client" code using the MPS generated classes.

2. "Server" code responding to MPS calls.

In 2, do you mean calls generated by MPS?

Regards,

Konstantin

Ran,

Querstion 1:

How would the client (java) use it?

1. languageProcessor.process("select ds1.attr1, ds2.attr2 from ds1, ds2;"); ?

2. Select.withDataSource("ds1").andAttribute("attr1").andDatasource("ds2").andAttribute("attr2").from("ds1", "ds2");

3. Some other way?

The first way won't be possible in MPS since it doesn't provide a parser. We use projectional editing approach which means that we work with AST directly. The second point approach looks like code which MPS can generate from your language but you will get no benefits from MPS by using it this way since you write it in Java.

In MPS you may define some queries and generate classes from them. You might get something like this:

class MyQueries {

   public static Query myQuery1() {

      return Select.withDataSource("ds1").andAttribute("attr1").andDatasource("ds2").andAttribute("attr2").from("ds1", "ds2");

   }

}

Regards,

Konstantin

Ran,

Are you familiar with Model Driven Development? In it you transform your

models (i.e. UML class and potentially sequence diagrams) to some sort of

code (java, xml, c# ...) which is typically utilizes some sort of API for

example if you take JPA or JDO you can generate a very complete persistent

domain model from UML class diagramm against JDO or JPA or some sort of DAO.

The output will include interfaces, implemenmtations for your domain model

xml metadata if any, some factory classes or hooks to IoC, you could

generate RESTful web services ....

you will use predefined transormers/templates or write your own.

also note that from one model you could generate it for different target

APIs (i.e. JPA vs JDO) or languages Java C#

DSL is one step above in generalization above graphical models like UML I

guess.

What bugs me is that i can not come up wit a really compelling and NOT

ARTIFICIAL case for using DSL.

MPS would go long way if it had a non trivial commonly useful example. I

doubt the product will make a hit on techical merrits unless there is a

really strong conceptual + implemantation example

Maybe a persistent domain object model with web services (or RESTFul web

services for CRUD operations) and + generic GUI generation perhaps

"Ran Biron" <no_reply@jetbrains.com> wrote in message

news:25883282.161821248725143045.JavaMail.clearspace@app8.labs.intellij.net...

Ok - I'll try a different approach.

>

Let's say I've developed a genius langauge for querying data sources - GQL

(genius query language).

Example of a query:

"select ds1.attr1, ds2.attr2 from ds1, ds2;"

>

Let's say I've modeled the language, created generators, editors and

what's not (I'll get back to this in a bit), generated the java classes,

packaged them into a jar and shipped to my client.

>

Querstion 1:

How would the client (java) use it?

1. languageProcessor.process("select ds1.attr1, ds2.attr2 from ds1,

ds2;"); ?

2.

Select.withDataSource("ds1").andAttribute("attr1").andDatasource("ds2").andAttribute("attr2").from("ds1",

"ds2");

3. Some other way?

>

Question 2:

Now - what would happen when the client does whatever he needs to do in

Q1? What part would he talk to (generator? actual language structure?)

>

Question 3:

How do I hook my genius backend processor to the langauge frontend?

>

---

Original message URL:

http://www.jetbrains.net/devnet/message/5242467#5242467

Hey Ran,

Instead of working on a text level, MPS works directly against abstract syntax trees.  MPS is not a lexer/parser, and would not (directly) store the query in your example as text.

I think that is simultaneously its biggest strength and weakness. 

You can do really neat and useful things I think that people haven't even begin to explore yet when freed from the shackles of syntax.

However, unfortunately, there are hundreds if not thousands of text editors out there, and only one MPS-compatible AST editor I know of, which is MPS itself.

I would take a look at ANTLR, if your needs involve parsing text into a model - ANTLR kicks some serious butt when it comes to making grammars.

That said, I do think MPS has some huge potential to do amazing things that a lexer/parser could never hope to!  Parsing text grammars just isn't one of them... :-)

Regarding the "MyQueries" part - that would force my client to define the queries in MPS and use it to generate the code, no? Is there some "standalone compiler" that might do the job (or better yet, an "online" compiler that can be used inside a java project?)

What I don't get is what can you do with your DSL after you've generated it? I mean,  writing something as compelx as a DSL to represent some internal structures and using it as a "one shot", where only I (with the MPS) can create queries for, seems a wasted effort. I'd better use regular models and be able to expose this to my entire team / project / clients.

On the other hand, writing a DSL that connects between my internal concepts and some business level concepts in a structured way (think "SDK") - that's something much worth the effort.

Please see the answer I posted on the thread AlexR replied to.

Hey Ran,

For that scenario, I don't think MPS is what you are looking for... if your language must be consumed by a mass audience that is already using Java, writing a set of classes in Java and possibly a DSL written with something like ANTLR sounds like a more natural fit.

If you ever have run into scenarios where subclassing/inheritance just aren't good enough abstractions for what you need, and you find yourself getting lost in the noise of boilerplate code, or writing something as a bunch of method calls just isn't a natural fit, or you have some syntactic sugar that would sure be helpful to readability and clarity in your code, etc... MPS will shine here.

So how can I use the langauge I create in MPS outside MPS? Please note I'm saying the language, not the saved queries.

Hello, Ran,

You can use language created with MPS only within MPS.

Regards,

Konstantin

Than the best thing MPS can be is a macro processor?

What's the point of building a langauge in MPS if I can't use it anywhere else? How would I ship that language to my customers? How would they use it?

Hello, Ran,

Than the best thing MPS can be is a macro processor?

MPS is IDE for lanuages which are created with it. You can do almost everything with and for your language there.

What's the point of building a langauge in MPS if I can't use it anywhere else? How would I ship that language to my customers? How would they use it?

Actually you can. There are two available options now for language distribution (you can read more about them here: http://www.jetbrains.net/confluence/display/MPS/Build+languages);

• you can package your language into special kind of archive and use it in other instance of MPS
• you can create your own MPS distribution with your languages included.

In the future we will have yet another option: ability to create IDE only for your language with language development, etc UIs removed.

Regards,

Konstantin

While I recognize the value of MPS, I kind of agree with your sentiment.  It seems to me that one of the most powerful uses of a domain specific language would be to make your domain layer something that could be utilized via DSL queries.  This would be particularly interesting for a SOFEA architecture, because within the web application, you could fully specify (in your DSL) the behavior that the web application requires of the server.  The granularity problem disappears.

This effectively makes your service an easy-to-maintain, single-layer system with a cohesive parser interface, whose design is driven by the desire to write queries in a clear and concise way.  The ability to submit transactional queries in an ad hoc fashion makes the code very exercisable and testable.

An interesting side effect of this approach is that rather than configuring access-controls on particular service methods, you need to implement access controls on data.  Information ownership becomes a first-class feature.

While the current incarnation of MPS wouldn't be useful for such a system, I wonder if a future incarnation could be.  Am I off my rocker?

Exactly. I can't ship out MPS. I can't convert my entire development base to an MPS created language - what I would love to do is craft a language in MPS that works as an interface to an "engine" created in another language (i.e. Java). Then I'd be able to allow my users, or even myself in my configuration builder hat, to use a sensible, domain specific idioms to control my software.

(Bump, on the off-chance that you'd want to evaluate the contents of the sister thread.)

Yes, but using MPS negates using my product - I can't embed MPS (and I don't want to).

Hi Ran,

maybe I'm a little bit late with my answer but perhaps you will read it anyway. So let me just give my two pence:

• to me it looks like you are not really familiar with the concept of DSLs (as somebody mentioned before); if you want to learn more about DSLs in general, you may look at Martin Fowlers DSL Work In Progress Book: http://www.martinfowler.com/dslwip/
• also generally and not MPS specific spoken: until now it was indeed (sadly) so that DSLs are often (and best?) used for the "one shot" approach because there were simply no tools available to do a full application generation; but what you can do is to strictly separate generated and hand written code (e.g. different jar files), then you can develop your DSL and hand written application code in parallel; Markus Völter had written lots about this approach, you may google for it
• regarding to MPS, it seems to me, your misconception is the following:
  • you develop your DSL in MPS
  • you think you have to use MPS at runtime in order to use the DSL output/generated things/stuff - because to interpret the DSL scripts at runtime just like SQL in that form: the "client" (your hand-written java code) tells MPS "hey, here you have that string which is something what can be interpreted as my DSL, please execute it and tell me the result" - just like SQL: tell the SQL Server (or JDBC driver) a string and get database rows as result
• well, but that is entirely not like it goes, its more like this:
  • you develop your DSL in MPS; that means you define a Meta-Model, that means a grammar in which you can write something thats correct regarding to this DSL; the same thing is true for Java (Sun defined the Java grammar), natural English language and so on
  • your DSL should not be really complex, but stay focused and small
  • then you use your DSL to generate tools like an editor; the tools are also used inside MPS
  • you use the generated tools to write something that is correct regarding to this DSL (our professor kept telling us like this: "you write words and sentences in this languages"); you do this in MPS, it's named a "solution"
    
  • you use a generator to transform the written something into other things you need; for now in most cases this would be Java classes, spring, struts, hibernate, ... configuration files, database creation scripts and so on; THAT is the really OUTPUT of MPS, just like Konstantin wrote in his first answers; the output is the stuff that leaves MPS' internals and what you can package, ship, or use otherwise
  • once you've got your MPS outputs, you don't need MPS anymore, you can compile the generated Java classes via javac, package the generated spring, struts, hibernate, ... config files directly, execute the generated SQL scripts using your favorite DB client; its just like programming with Eclipse: once you've written your Java code and compiled it, you don't need Eclipse at runtime to execute your Java, you don't have an "Eclipse client" which calls Eclipse with: eclipse.execute("public class MyClass {....}")
    
• the clue of using MPS or any other MDA/MDSD tool (like openArchitectureWare, MetaEdit, androMDA) is that you have to write MUCH less code as DSL "scripts" and generate the stuff as you would have to write by hand; e.g. you have a or some cool DSLs and write lets say 1000 lines of "script" code with these; then you generate 100000 lines of Java and XML code out of the DSL code; now lets think about how long it takes to write 1000 lines of code and how long it would take to write all these 100000 lines of Java and XML by hand
• another great thing about DSLs is that you can write the DSL "script" code much more readable as you could do it in Java or any other general purpose language; thats even true for languages like Ruby where it is easier to write readable "internal DSLs" (see Fowler for "internal" and "external" DSLs)

Best regards

jens

Wow. Great answer.

So, in essence, MPS is similar to a script generator (a "scripted" generator?). Its output are intermediate constructs - not something that can be shipped to a customer as is. They need to be wrapped, executed, parsed, compiled...

Don't get me wrong - I'm not belittling it. Generators have their place and use. Indeed, I was under the misconception MPS is like IDEA - an IDE that is used to create end products. I think this distinction should be highlighted on the main page of MPS (probably with a link to your post - it's THAT good :-)).

Now that I understand what it does - where can I find well built examples of langauges, use case stories, etc? There are some areas in my code that could benifit from being automatically generated.

Hi Ran,

well, also IDEA output is in most cases not ready to be shipped - often tools like ant and maven are used to package it, transform XML config files, create version numbers and so on. And so the MPS output is: in best cases the MPS output is nearly the end product. Jetbrains does this with YouTrack/Charisma. But as it happens, DSLs are very old and wide spread, but the everybody-creates-her-own-DSL-hype is just lifting of these days. And so full generation of products is not wide spread, but with tools like MPS it will become much easier and common. State of the art for now is indeed a partial generation and partial hand writing the code.

I compare DSLs these days with design pattern and refactoring in early days - not much tools, not much literature, not so good common understanding how to do it. But if you google a bit, you'll find plenty of examples and articles, mostly using oAW. You simply have to adopt it to MPS ;-) For MPS I don't know production success stories or other war stories/languages except the ones from Jetbrains itself.

Perhaps we could do something in the community using the wiki. (Yes, there is a MPS wiki, but unfortunately its not mentioned on the docs page and only with little content for now.)

Best regards

jens

I know this is an old post but this post explained MPS for me better than the product homepage. Thanks for the article.