DSL Testing

The testing depends on what you test.

• constraints: while it's not possible to check completion menu, still if you want to test your constraining code (not how MPS completion menu works), you can make a test with a "bad" model and check whether MPS reports some broken constraints (like the one on the screenshot, screen is from 3.2 EAP)
  
• textgen: I think the most reasonable way to check textgen is by generating models for which we already know correct generation result, like said in the topic you've referenced.
  
• migration: you can do it just by extracting a method that migrates one node in the migration and using it in a "node test case", for example.

In common, you are right that now there's a problem with the whole system: there's no "universal" way to test any aspect and no tutorial.  We'll think about what we could do here. If you have any ideas on that, don't hesitate to propose any.

You may have found http://confluence.jetbrains.com/display/MPSD31/Testing+languages already. This is all there is at the moment to testing in MPS. We will try to add more details on testing.

All tests are performed within your browser over HTTP (TCP) and require no further software other than a newer browser with activated Flash.

Thank you for your reply and sorry for my delayed reply!

I absolutely agree with you referring to your statement about the dependency between "what to test" and "how to test". Referring to your examples, I suggest the following:

• constraints: I do not want to test whether the MPS code completion menu works as desired. Instead I've been thinking of some kind of test language to check if my constraints work. By just creating a "bad" model one would encounter the issue that he or she has to remember the constraints in the model. Using a dedicated test language it would be possible to explicitly write down the expectations the language author has. Additionally the author could run all test cases for the language and it would be possible to check language correctness at a glance. Of course the latter depends on the completeness of your test cases. Stepping over the different statements of a constraint while executing the test would also be very helpful.
  
• textgen: It is almost the same argumentation as for the constraints aspect. By providing a test case that checks whether the textgen transforms node<PublicVisibility> to "public" one could again provide expectations. The notation could be very easy, e.g.

node<PublicVisibility> yields "public"
condition: <<always>>

Maybe there is a more expressive or useful notation.

• migration: Ok, this would be possible, I think.

If I can help with some requirements for testing, don't hestitate to ask me. I'm only one MPS user but my requirements may be a useful starting point.

Thank you again and kind regards,
LaAck

Yes, thank you. I already found this documentation. It is helpful but not to the extend, I currently need.

Could you provide more details to explain what you mean, please? I don't get it, yet.