Introducing the new client-server protocol

It may seem that progress on this project has been quite slow during the last few weeks. Partly, this is because progress has been quite slow during the last few weeks. Writing a lot of self-contained code is easy, but as soon as I have to interface with the Eclipse API, each line of code can easily take an hour of poking around in documentation. Another reason for the lack of visible progress is that I’ve spent a few days refactoring and reorganizing the preference pages for EclipseFP. However, these changes aren’t ready yet and are currently parked away in a branch. But I decided not to let work “under the hood” keep me from posting updates!

Nominolo (my mentor, and the author of Scion) and I decided that it was no good to have the Scion server speak different protocols for each type of client that it serves. There were already separate Emacs and vim protocols, each with a different set of accessible commands, and adding and maintaining additional protocols would quickly grow unwieldy.

We therefore settled on one, and only one, protocol that is easy to parse from nearly every language: JSON. As simple as possible, but not any simpler. I threw away my hacked Lisp parser and implemented the JSON-based protocol in EclipseFP.

There are several options for JSON parsing from Java. Two feasible ones referenced from the JSON homepage are org.json (or whatever its name is) and json-simple. org.json uses specialized objects to represent the different JSON types, whereas json-simple uses standard Java classes. Although this sounds simple and elegant indeed, I abandoned json-simple because it does not employ generics, and therefore still requires many typecasts. In fact, neither of the two libraries are very nice and type-safe (because of JSON’s dynamic nature, they cannot be!), but org.json provides many utility methods that save typing out explicit casts.

Apart from simplicity and well-definedness, JSON has another advantage over the previous protocol: it is now possible to test the Scion server through telnet.

Posted in EclipseFP. Tags: . 3 Comments »

Cosmetics

Since I was feeling creative today, I decided I would work on some cosmetic issues in EclipseFP that have been bothering me for a while. They make the whole thing look unprofessional and out of tune with the rest of the Eclipse environment. Luckily, Eclipse has a User Interface Guidelines document for me to follow.

I started with some icons. The Guidelines specify precisely how they should look. The icons used in the Package Explorer are now in the same style as their Java counterparts:

project-explorer

Note how the Haskell source files have a nice lambda icon, and the literate source file also has its own icon. The Haskell project itself has a subtle lambda overlay. (I see now that I missed the Libraries icon. Also, the text “[GHC]” should be dimmed. Made a note of these.)

Editors, too, get the new icons:

editor-icons

And finally, the one that I’m most proud of. I figured it would be nice to have the new official “lambda-and-bind” Haskell logo somewhere. And what better place for this than the icon of the Haskell Perspective? So I brushed up my pixel painting skills and drew a variant of the Haskell logo that blends in with the Eclipse environment:

perspective-icons

It’s always there, subtle, but visible, recognisable and fresh. I like it.

Posted in EclipseFP. Tags: . 3 Comments »

Public code release

These instructions are outdated. See the new instructions!

My code is now in a sufficiently clean state that I can push it out to the world. For anyone who wants to give it a try, here are the instructions that should get you up and running. This assumes that you have GHC, cabal-install, darcs, and git. I tested only on Ubuntu 9.4, but there is no reason why it should not work on Windows as well.

  1. Get my branch of the Scion sources:
    git clone git://github.com/ttencate/scion.git
    (It should also work with nominolo’s original sources.)
  2. Build and install Scion, the library as well as the server program:
    cd lib
    cabal install
    cd ../server
    cabal install
  3. Get Eclipse Galileo (3.5 RC4 or later), the distribution named “Eclipse for RCP/Plug-in Developers”. Extraction equals installation.
  4. Get my branch of the EclipseFP sources:
    darcs get http://code.haskell.org/~ttencate/eclipsefp/
    darcs get http://code.haskell.org/~ttencate/cohatoe/
    (Although Cohatoe is probably going to be removed, this version of EclipseFP still depends on it. Build the Cohatoe server if you like; it segfaults on my system.)
  5. Import all Eclipse projects from these repositories into Eclipse. There are over forty, so the Multi Project Import/Export plugin might come in handy.
  6. Wait for the build to finish, then… Run!
  7. In the new Eclipse window that (hopefully) pops up, set the location of the Scion server via Window, Preferences. If you installed it locally, it should be in ~/.cabal/bin/scion_server. (Eclipse does not understand the tilde, so give the full path to your home directory.)

It’s quite some work to get it all up and running, and I probably forgot a step or two. Making the process easier is not currently a priority, but I envision a complete Eclipse release that is easy to set up and has everything you need to get Haskelling quickly.

If you try these steps, and run into any problems, please let me know, so that I can help you out, and improve the instructions and maybe even the code at the same time.

“Open Definition” and code improvements

Work on this project has been slow until a few days ago, but now I’m back in business. I’m getting the hang of this Eclipse API thing, which makes everything go much smoother. Today I present to you: a more robust Scion client, a rudimentary “Open Definition” feature, and a configurable path to the Scion server program.

The improved client code for the Scion server works with a command queue. It spawns one thread that is tightly coupled to the server process. This thread waits for a command to arrive in the queue, then sends it off to the server and waits for the response. If the server crashes for some reason, it will be restarted and the command will be retried on the new incarnation. The original sender of the command (for example, a GUI element) specifies a timeout: if the command is not executed successfully within this time, the client gives up and the original sender is left to deal with the problem in whatever way it sees fit.

Then, for the mandatory screenshots, I built a feature similar to the “Open Declaration (F3)” feature that you might know from Java development in Eclipse. You can select an identifier and hit F3 (or go through the context menu or main menu), and you will jump to the place where the identifier is defined:

A context menu item to open the definition of the selected identifier

After clicking Open Definition, we are taken to the definition of the selected function

This feature is very rudimentary: it only works if the definition is in the current file; you have to select the identifier; and the Ctrl+click that Java users may know does not work yet.

I also added a Preferences page to configure the location of the Scion executable. Autodetection is in the works, but is not there yet. Automatically installing Scion via Cabal upon request would also be a very nice feature to have.

This preference page may be a trivial feature, but very important: now that the path to my home directory is no longer hard-coded, other people might actually be able to try my code too! Feeling adventurous? See the next post!

Client/server communication

Because of other obligations (mainly my Master’s thesis) I had a late start with this Summer of Code, but now it has really begun. Below are the first results, but first some background information.

To make Eclipse understand your Haskell source, a lot of parsing and processing code is needed. Fortunately, this code has already been written, in the form of the GHC compiler. Unfortunately, this code is written in Haskell, whereas Eclipse is written in Java. Part of the purpose of Scion is to overcome this problem. On one side, it communicates with the GHC API; on the other side, it communicates with whatever program is interested. This used to mean Emacs only, but now it also includes Eclipse!

There are essentially two ways to make Scion communicate with a non-Haskell program. One is through an intermediary C program. Haskell can call into C using FFI, and Java can call into C using JNI, so this is technically possible. However, it would be very painful and cumbersome.

The alternative is to communicate through some kind of intermediary format that is sent over a stream (say, a pipe or a socket). This is how Scion’s current Emacs frontend is implemented. When needed, Emacs launches a separate process (scion_server) that listens on a local TCP socket. Emacs connects to the socket and sends a command; the server responds in a similar, Lisp-like format. (In fact, the returned value is sent straight into the Lisp interpreter.)

To do a similar thing with Eclipse, I could either make the server speak another language that is easily parsed from Java (XML-RPC comes to mind), or make Eclipse parse the Lisp code. To get quick results, my mentor nominolo and I chose the latter route, so I hacked together some code to parse a small number of specific responses. The full Lisp parser will have to wait a day or two.

Add some code to start the server, connect to its socket, send and receive commands, and we’re all set. The screenshot below shows one of the many things that are now within reach: automatic type inference, in a tooltip, straight from Eclipse!

A tooltip showing the inferred type of a function

The current code is still full of TODO comments, lousy error handling and hard-coded configuration, but it works. At the moment it is necessary to save the file before the type information gets updated, but I hope that it will be possible to send changes to the server incrementally.

Over the next few days, I will replace my quick hacks by something better, improve the error handling, and make the whole thing more robust. And I should also really, really do something about the hard-coded absolute path that points to the Scion server in my home directory…

Posted in EclipseFP. Tags: . 3 Comments »

Project description

In this post, I will describe what my project is all about. What is it that I intend to do, why do I intend to do it, and how?

Haskell is a pure functional programming language. Pure functional means that the entire program is treated as the evaluation of an expression; the word pure indicates that such an expression can have no side effects. Hence, no modification of variables, no reading from disk, no drawing to the screen. Of course Haskell wouldn’t be very useful if it couldn’t do all those things, but indeed it can. The language is now growing out from its origins in computer science research, as more and more people begin to realize that purity and strict typing are good things. Although the initial effort that goes into a program may be a bit more than with more mainstream languages, these language features guarantee a certain degree of correctness that impure, dynamically typed, or imperative languages can never achieve.

However, I believe that the further acceptance of Haskell is held back by a lack of tooling. There is an great compiler, great libraries and a great package management system, but there is no really great IDE (Integrated Development Environment). A good IDE would not only be a great time saver for seasoned Haskell developers; it would also be of much assistance to newbies who want to learn the language.

There is Leksah (Haskell spelled backwards, missing an L), but it is still under heavy development and not yet in a really usable state. Yi is an editor written in Haskell, but it does not qualify as a full IDE. Either way, there is no reason to build yet another IDE from scratch when a perfectly good option is readily available.

Eclipse is one of the most popular IDEs  for the imperative programming language Java, and it is also written in Java. I worked with it years ago and hated it. It was unwieldy, incomprehensible and terribly slow to boot. But recently I was forced to work with it again, and I was pleasantly surprised. Because it is a very powerful IDE there are still many knobs and dials, but they are generally in sensible places and can be understood without a manual. The speed has also improved greatly. But the best thing about Eclipse is its feature set. It has clever code completion. It has a powerful build system. It has refactoring features that outdo even the latest Visual Studio releases from Microsoft. And it is not just about Java: rumour has it that Eclipse can also be used for C++ development, PHP… and Haskell.

Indeed it can. EclipseFP (which stands for Eclipse Functional Programming) is a plugin for Eclipse that allows for Haskell development. (It also contains discontinued support for OCaml.) Unfortunately, EclipseFP is only marginally usable, does not offer half as many useful features as it could, and has been standing still for almost a year. I figured that’s a shame, so I decided to try and see if I could improve this situation. Thomas Schilling a.k.a. nominolo has graciously offered to be my mentor, for which I am vary grateful. Leif Frenzel, the original author of EclipseFP, has also offered his help, so I am in good hands.

I will start by fixing small bugs and making little improvements here and there. This way, I will familiarize myself with the source code of EclipseFP and with the Eclipse plugin system.

Next, the real work will start. The current EclipseFP does its own parsing of Haskell, which is a duplication of work: there is already a perfectly good Haskell parser and typechecker available in the form of the compiler, GHC. The Scion library was developed (by the aforementioned nominolo) specifically for interfacing with GHC. Thus, my next course of action will be to hook EclipseFP up to Scion.

This opens up many exciting possibilities. The type of an identifier could be shown in a tooltip when you hover the mouse cursor over it. Inferred type signatures could be automatically added to functions. Type errors and other compile errors could be highlighted with red squiggly lines. It could be possible to Ctrl+click on an identifier to jump to its definition.

Near the end of the project, I will add support for the Haskell package manager and build system Cabal. EclipseFP currently uses its own build files, but using a Cabal file would greatly simplify the transition to Eclipse + EclipseFP and also allow for easier integration with existing libraries. Some IDE support in writing the Cabal file would also be great.

So, that’s the plan for the next months. It will be a bumpy ride. It will be difficult at times. But I’m definitely going to learn a lot, and will hopefully even turn out a useful product!

About me

I figured that the readers of this blog might want to know some more about me. If not, you can skip to the real stuff.

My name is Thomas ten Cate. I’m a 24-year-old Computer Science student at the University of Groningen, a wonderful town in the north of the Netherlands where I also live. I am about to finish my Master’s degree in CS, and I also have a Bachelor’s degree in mathematics.

My programming experience includes a lot of C++, a lot of Java (mainly for teaching) and also a lot of C#. I was taught Haskell in a university course on functional programming years ago. Unfortunately, that course did not go very far, so I have been learning me a real-world Haskell since. I cannot call myself a Haskell expert, but I am beyond the beginner’s level. For example, I do grok monads, but am still hazy on stuff like functors and arrows. I hope to learn much, much more about the language during this Summer of Code project.

Other experience includes half a year at the Interactions Lab at the University of Calgary, Alberta, Canada, which gave me much valuable insight into human-computer interaction that will be useful when modifying and extending the user interface of EclipseFP. Furthermore, I run a one-man open source project called Taekwindow, which you might like if you are occasionally forced to use Windows.

I chose to do this Summer of Code project because I wanted to write something more serious in Haskell than the small programs that I’d toyed with so far. With that desire came the need for a good IDE. Kate is a nice editor, but it will not compile for you, or highlight errors, or run your program. I poked around the web and found that I seemed to be desiring something impossible. Then the Google Summer of Code came along and gave me an opportunity to try and make the impossible possible, so here I am!

Posted in EclipseFP. Tags: . 1 Comment »
Follow

Get every new post delivered to your Inbox.