Syntax-Highlighting Experiments

Published on Mon 01 July 2013 in Nepomuk, (Comments)

Sometimes, I’m very happy to work with Qt, because it provides everything I need, and in a very clever way. Syntax-highlighting is one of the many areas where Qt shines.

High-level syntax-highlighting of programming languages and configuration files is easily done using KTextEditor (the component used by Kate, KWrite, Kile, KDevelop, I think). One has simply to write an XML file describing the language to highlight, and the editor does the rest. Code-folding is also possible, as is every other feature everyone loves in Kate and KDevelop.

For my Google Summer of Code project, such a full-fledged editor is a bit overkill. What I need is a mean to syntax-highlight a one-line text edit. The highlighting itself is simple, because the “grammar” (if it really is one) of the parser is simple.

A One-Line Text Edit

Yesterday, I started by implementing a small editor based on QPlainTextEdit, mimicking QLineEdit. I couldn’t directly use this class because it doesn’t allow syntax-highlighting, I need a QTextEdit subclass and a QTextDocument. I therefore tried to make QPlainTextEdit look like a simple line edit. This can be done by configuring it properly (setting the size policy, disabling tabs, disabling the scroll bars and the word-wrap, etc) and by reimplementing its sizeHint() method. The reimplemented method behaves like the one of QLineEdit (original code from the Qt project FAQ):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
QSize QueryBuilder::sizeHint() const
{
    QFontMetrics fm(font());
    QStyleOptionFrameV3 opt;
    QString text = document()->toPlainText();

    int h = qMax(fm.height(), 14) + 4;
    int w = fm.width(text) + 4;

    opt.initFrom(this);

    return style()->sizeFromContents(
        QStyle::CT_LineEdit,
        &opt,
        QSize(w, h).expandedTo(QApplication::globalStrut()),
        this
    );
}

The method calculates the area of the displayed text (using QFontMetrics), and then asks the style to compute the size of the whole widget. By telling the style that the widget is a CT_LineEdit, the size returned is the exact same size a line edit would have. The illusion is perfect:

Query builder widget

Choosing Colors

After having implemented the text editor, I started experimenting with the syntax-highlighting itself. My idea was to have every ComparisonTerm displayed in a different color, with the literal terms displayed in bold. During the development, I also thought that it would be nice to have “type hints” (mails, photos, documents, files, contacts, etc) displayed in italics.

My first idea was to follow the interface given by Ivan Čukić here. The problem was that I don’t even know how his “single-line edit with grouped terms that can be removed by clicking on a cross” is officially called. So, I’ve found nothing on Google that could help me to implement that. I tried different solutions, but there were always problems (for instance, a QLineEdit draws its white back-ground and its text in one step, so I cannot draw boxes between the two). In order to still be able to experiment things, I decided to use a simpler approach (syntax-highlighting) for now.

In the above image, you can see that every comparison term is highlighted in a different color. The colors are coming from the Oxygen palette, and are roughly all of the same luminosity. It is very difficult to find colors that look well with each other, so if you have ideas, don’t hesitate to tell me. I have selected 8 colors, and the highlighter cycles between them.

I also tried to use HSV colors (Hue, Saturation, Value). Saturation and Value were forced to values often found in Oxygen colors, and the hue was incremented by 71 degrees every-time the color had to change. The result was ok, but not so nice. Sure, there was up to 360 different colors, but they were mostly bad-looking and difficult to read. I abandoned the idea and decided to use Oxygen colors, that were carefully crafted by the great designers of KDE.

I developed this experiment directly in the Nepomuk Widgets repository. As I have no KDE developer account yet, the repository can be found at http://public.steckdenis.be/git/nepomuk-widgets . My code is in the gsoc2013 branch. Feel free to experiment and to modify the colors (a table in ui/querysyntaxhighlighter.cpp).

UPDATE:

Most users will never or very rarely use comparison terms. All they want is a tool that returns a list of documents matching terms. The syntax-highlighter I presented in this blog post chooses a different color for each comparison or literal terms. Queries like “cat dog rabbit” therefore are highlighted using 3 different colors, every term being in bold. This is not really nice.

So, I changed the highlighting a bit. Now, literal terms are not highlighted at all, only comparison terms and resource type terms (type hints) are. For comparisons, the whole comparison is colored, and its part that is not a literal term is also rendered in italics. So, “foo bar sent by Jimmy” becomes “foo bar sent by Jimmy”:

Don't highlight literal terms

Another modification is that nested queries are detected and underlined, so the user can see where it ends, and potentially why its query does not return the expected results:

Underlining of nested queries

« Merging the parser   A Nicer Query Builder Widget »