copy/paste text in MPS editor

Tom,

There is special merge support as Konstatin mentioned in this thread :

I have not used this merging support yet myself, but I assume it is the same as when you look at the changes for a specific file. In the Changes pane you can show the diff of e.g. editor.mps. This diff overview shows changes to the underlying XML as added nodes, changed nodes, etc. instead of simply text based changes.

Ruben

Thanks for pointed to how VCS merges are handled, Ruben. This seems to be an adequate way.

My mainpoint ("copy/paste plain text from other sources is a very important feature for usability") is still open, though. I was surprised by the total absense of reactions to my post above. What should I conclude from this silence? Is this a fundamental limitation that everybode accepts. Am I the only one missing this?

-Tom

Tom,

I totally agree that while in developer mode not having the freedom of working with plain text feels akward at best. But when it comes to exposing the DSL  to an end users without any tachnical/development background, I consider it a strength of MPS that no syntactically invalid content can be entered in the editor.

Ruben

Ruben,

I do not fully agree with the point you are making. Let me try to explain:

I see the advantage of limiting the (end)-user so that he is only able to enter syntactically correct expressions and helping him to do so extremely well,... but... even in that setting it would be an advantage to be able to paste raw text. Imagine the situation where two users exchange some contents via mail, or some other text based communication system. Suppose one user gives a snippit to the other how certain functionality can be defined. The other now has to tediously go through that snippit item-by-item and find the right entries in the popup menus. Imagine this to be a few hundred lines and you will see the absurdity.Exchanging xml is not what I would like as a developer, forcing my users to do so would be out of the question.

BTW: I discovered this lack of functionality because I was doing the tutorial and had to key in several multi line snippits that I would normally copy/paste in tutorials. Since my mind is always on automating manual labor ;-) I looked for a way to copy/paste and did not find any.

-Tom

Maybe a combination of MPS-readable data and text is a possibility? Imagine sending something like this. The block of MPS data is automatically prepended by MPS when you copy a piece of code to the clipboard. MPS has to store the data on the clipboard in multiple flavors (MPS format, plain text and rich text) so it can be pasted in both MPS and text applications:

{MPSsnippet:base64 encoded

compressed data

goes here}

public class Foo extends Bar implements Baz {

    public static void main(String[] args) {

        System.out.println("Hello, world!");

    }

}

The MPS data is easy to ignore because it are just a few lines of text. And with rich text it can be given a light color so it doesn't stand out.

If a user wants to add the code to his code he only has to select the MPS snippet, copy it to the clipboard and paste it in MPS. He might not want all of the code, but this is easily solved by letting the user choose what part to paste in his editor. And MPS can highlight which parts can be pasted in the current editor cell to help the user decide.

Note that MPS also has to translate the presentation in the editor to plain/rich text. For plain text this is difficult, and for rich text it might not be possible and/or be too expensive (in terms of the amount of resulting rich text) to recreate the editor presentation.

Regards,

Johan

This might be a solution, but would it not be possible to parse the plain text and deduce the meaning of it in the current MPS-editor? The editor knows the syntax after all, it can currently generate the textual representation that resembles the presentation on screen. Is there any fundamental reason why this process can not be reversed? When I key in the snippits from the tutorial, as a human I have no problem interpreting the text. I would say that at least an automated educated guess could be made what the MPS-editor intrepretation would be. It would be acceptable to have some red patches where a manual resolution is needed.

Any ideas or clarifying remarks from Jetbrains?

-Tom

Hi Tom,

parsing the plain text may not be sufficient, because it is most likely

ambigious as multiple mps concepts may have the same textual

representation (meaning concrete syntax). That's why MPS serializes the

AST as XML and not plain, human readable text.

Regards

Sebastian

Tom Brus schrieb:

This might be a solution, but would it not be possible to parse the plain text and deduce the meaning of it in the current MPS-editor? The editor knows the syntax after all, it can currently generate the textual representation that resembles the presentation on screen. Is there any fundamental reaso

n why this process can not be reversed? When I key in the snippits from the tutorial, as a human I have no problem interpreting the text. I would say that at least an automated educated guess could be made what the MPS-editor intrepretation would be. It would be acceptable to have some red patches w

here a manual resolution is needed.

Any ideas or clarifying remarks from Jetbrains?

-Tom

---

Original message URL: http://www.jetbrains.net/devnet/message/5230958#5230958

Here is an extreme example of the problem with parsing text. Suppose I develop the following four concepts:

- addition

- subtraction

- multiplication

- division

And I create editors for them that render like this:

- x + y

- x + y

- x + y

- x + y

These editors are totally useless of course, and you won't do it this way if you control all the concepts and their editors. But as soon as you start importing concepts from other people you are bound to run into these situations. MPS has no problem dealing with the situation because it knows exactly what is meand by each 'x + y'.

Automatically building a parser from the editor specification will be very hard and time-consuming, and I strongly doubt if the results will please a large percentage of users if it is attempted. The parsing will also become harder and harder as editors become more and more sophisticated. Suppose I integrate a funky editor to enter a 3D coordinate using advanced graphics and the mouse. If being parseable is a requirement I also have to write code to convert the value to text, and a parser for that text. And what about the view state of the editor? If a user copies the editor, does he expect the exact same view will appear when he pastes it into another file?

I hope this clears things up a bit and helps you understand why the MPS folks avoided parsing.

Regards,

Johan

I see your point about ambiguity. We humans can cope with just a moderate amount of ambiguity, if expressions are getting too ambiguous we give up (as your example illustrates). So sensible editor designers will make sensible editors that do not have a lot of ambiguity, otherwise they become unusable. When this is the case I do not see why it is impossible to derive a parser from the editor definitions (although it may not be easy, I admit). When confronted with ambiguities this parser can ask the user to clarify the situation. In my opinion this would greatly enhance the usability of the MPS editors.

I take the example of the tutorial again: the text representing java is not ambiguous at all, that is why I can key it in without any problem (except getting blisters on my fingers ;-)). For the same reason I can imagine a parser that determines what is represented and translate it to the underlaying AST.

The possibility that ambiguous syntaxes can be defined should not hold you back to making editors more usable if there is no ambiguity at all. The only problem could be that it is not computable that there is ambiguity in the editor definitions. As far as I know this is not the case, but correct me if wrong.

Some problems I do see:

1. there is more info in the AST then there might be visible in the editor. If you copy/paste just the visible info you are missing information.
   
2. when you paste some perfectly well formed text at a place where it does not fit in the syntax you might invalidate the whole editor contents.

As of yet I have no ideal solutions for these problems.

What about having two modes in the editor: one syntax directed mode (as it is now) and one free-text mode where a parser is constantly trying to adjust the AST to whatever is typed. Both modes have their value and it would be even nicer if the editor switched modes on the fly. The free-text mode is actually what we use in InteliJ every day. It's parser constantly adjusts the AST on the basis of what I type. If I type something that can not be parsed it is indicated in the editor. Optionally the editor will help me to correct the illegal syntax. The intelliJ editor also has some advanced syntax directed features (as you probably know) that are helpful but not obstructing. I would like to see this same flexibility in MPS-editors.

IMHO this is the Achilles heel of MPS. I am an MDA evangelist, it is the way to push software development to a higher level. I am charmed by the MPS approach. When I write things from scratch I am very happy that the editor helps me pick the right constructs. But when I need to move code around, reorder and polish it, I want to copy and paste when I like it and not key things in again and again. When I have to, I get very aggravated. Imagine that you would have to write and maintain all your java code this way... I would happily switch back to vi ;-).

-Tom

BTW, we have plan to improve merge support so that you will be able to see changes in editor with added/removed/changed/confliced parts highlighted.

Hello, Tom.

You can copy code from MPS and paste in somewhere else but you can't do reverse. The reason is a complexity of parsing an arbitrary grammar which get composed from many other grammars so the resulting grammar can be ambigous. Yes, humans can parse a code in such a language when they look at it but they do so not only on a syntactic level. We use not only syntax but information about types, meaning, etc. Because there are a lot of aspects which can help us to do so, it's very hard to implement.

We make everything easier by storing a model in parsed form. We don't have to think about grammar ambiguities and other stuff. Being able to copy code between text and MPS will improve usability experience. In the future we might provide ability to implement such a support for user language's copy-paste from/to text but user have to create a parser for this. This thing won't be provided for free.

Hi, we were discussing the text pasting problem with Timur Zambalayev when idea of another possible solution has emerged.
This approach doesn't employ parser.

Suppose we want to paste text like this:

label.setText( "java" );

If we let MPS know about our intention, it won't convert this text into AST but it could provide much more assistance when we start typing (thus much less if not at all blisters:)).
For instanse, we type 'l' and MPS already knows that it is going to be 'label.setText(...' and it could try to 'playback' successive symbols pretty much as if it we were typing.

If indeed there is variable 'label' in context and its class has method setText() then MPS will be able to create this entire statement.
If MPS meet an ambiguity (for instance, if text 'label' matches several alternatives) then it should stop 'playback' and let us disambiguate, then go on.

Igor.

This approach has very limited applicability. Completion menu items don't have to match editor presentation. This solution will work only for items like limited subset of baseLanguages' expression.      

Hello Konstantin,

I understand that it is not an easy thing to accomplish and it might even involve special definitions (I guess you mean 'not generatable' when you say 'This thing won't be provided for free' and not 'non-open-source' ;-)). I think it would be worth the effort for langages that are widely used (like the java editor).

What I am curious about is if you agree that this is a serious usability issue. Since MPS is defined in itself you must have used the MPS editors extensively (right?). Please correct me if I am wrong but there must have been occasions where you thought: "I wish I could copy/paste this bit there...". Maybe you solved those by editing the underlaying XML directly?

Get me right, I am not trying to flame, I am trying to point out a shortcomming that I believe would hinder general applicability of MPS. Whatever solution is possible, albeit a partial solution, would greatly increase the survival changes of MPS. I am a fan of IntelliJ and Jetbrains, I think you guys are doing a very good job and I would love to see that quality move into the modelling arena because good tools are essential for a wide acceptance of model driven development.

-Tom

Hello, Tom.

I understand that it is not an easy thing to accomplish and it might even involve special definitions (I guess you mean 'not generatable' when you say 'This thing won't be provided for free' and not 'non-open-source' ;-))

I wanted to say that it won't be available without special efform from users not by paying money :-)

I think it would be worth the effort for langages that are widely used (like the java editor).

Creating a good parsers with AST for java is a complex task. We have plans to have IntelliJ IDEA plugin. IDEA already has excellent AST for Java. When we implement such a plugin, we will be able to paste Java code relatively easily.

What I am curious about is if you agree that this is a serious usability issue. Since MPS is defined in itself you must have used the MPS editors extensively (right?). Please correct me if I am wrong but there must have been occasions where you thought: "I wish I could copy/paste this bit there...". Maybe you solved those by editing the underlaying XML directly?

We have copy-paste from MPS to MPS. We also have copy-paste from MPS to text but we don't have copy-paste from text to MPS. When we develop something in MPS, we usually copy paste from MPS to MPS only.

Hi Konstantin,

We have plans to have IntelliJ IDEA plugin. IDEA already has excellent AST for Java. When we implement such a plugin, we will be able to paste Java code relatively easily.

This sounds great, covering java would probably gain a lot of ground (at least from my point of view, but I am probably slightly biased ;-)).

We have copy-paste from MPS to MPS

Oops, I did not realize that. This indeed makes live inside MPS a lot pleasurable. So this also means that while doing the tutorial I could have copy/pasted lines like

public void insertUpdate(DocumentEvent p0) {
    CalculatorImpl.this.update();
}

to quickly make the remove and change variants of this method?

This indeed limits the extend of my comment.

-Tom

I just went through the excruciating process of writing an array of 256 hex characters by hand because, even after 10 years, there is *still* no way of copying code from outside MPS to inside!!!