perjantai 16. huhtikuuta 2010

Application's element selection and traversal based on eye movement

A lot of today's computer usage involves moving between hypertextual links. Unfortunately mostly used method is mouse which puts stress on wrist. There are some alternatives, like providing keyboard shortcuts or combinations to select the wanted element but it's not optimal.

One could imagine instead a system where eye movement was tracked. This could be done for example by using lenses that are put onto eyes (similar to those used for enhancing eye sight). The problem is how a signal can be sent without annoying the user or other problems. An infrared solution is possible, perhaps.

What I find interesting and important for a cheap solution, is that most traversing across the screen is not needed to be very accurate. One can imagine that programs would expose their elements for the application-agnostic tracking softare, and it would pattern match to nearest element, depending on what mode user had put on by keyboards. Of course, this can be automatically context-sensitive when switching between programs. For example, when browsing, the important elements are html's reference and input elements.

For the selection action, a keyboard press, or a wink of an eyelid is sufficient.

torstai 8. huhtikuuta 2010

Factorscript - jQuery and Factor proof of concept

A couple of weeks ago I released first version of Factor playground, which is a Factor language learning environment. It contains a javascript interpreter which can be used for example to draw on HTML5 canvas.

I thought it would be cool to have an interface to jQuery from the Factor interpreter so that one could script whole web page without single Javascript expression. I made a crude attempt at it, which I'll now shortly go through.

My first problem was how I can include script without putting it into Javascript string, which would be cumbersome and ugly-looking. I accidentally read an article how John Resig had done in his microtemplate Javascript: just put everything inside <script text="text/html">

Web browsers will ignore that kind of tag, and it's easy to fetch the content with ordinary DOM crawling methods when the script is given a class or id attribute.

It also seems there's no problem with having many script tags with the same class file, and that the order jQuery fetches them is from the top to the bottom, so it is possible to create dependencies between multiple script tags. For example, I created a word for modifying the css property of a div that has class "box" and put it on top.

edit: I was informed that one could create her own type for script, e.g. "application/x-factor". Thanks Chris!

The (first) jQuery interface word I show is named $2. It basically takes four arguments, value, attribute, property and selector. Selector is a string (or jQuery object), property is method name, like "addClass" or "css", attribute in our case is "background-color" or "color" and value is e.g. "green". It's all very crude at this point; we can't even query for a value from an object, which needs a javascript-to-Factor-object-translation However, we can proceed now with a new word "box-css" that uses $2.

: box-css "css" ".box" $2 ;

Next, we can set the background-color and color property by adding string values to stack and calling the just made box-css-word.

"yellow" "background-color" box-css
"blue" "color" box-css

This will be transformed into simple jQuery call such as 

$(".box").css("color", "blue") 

and similary for background-color. There are also words for method calling that have only one or zero arguments, $1 and $ respectively. Instead of $, $1, and $2 there could be just $. It would take an array containing the arguments for jQuery call. I guess that would make the code more readable.

The interesting case is when we want to add an event to elements.We need to delay the interpretation of user specified computation, which in Factor is called a quotation. In our case we want to get a reference to the jQuery object $(this). We can achieve this by creating a quotation with named parameter that pops the value from stack which is put there by the interpreter.

[| this |
  "calculation_click" "addClass" this $1
] "click" ".calculation" $$
So in the above we have selector for class "calculation", and event named "click", and a quotation to be executed when user clicks suitable element. Notice we have used $$ to bind the event with quotation but we also have another jQuery call inside that quotation. It takes "this" as parameter but it can be named however one likes, since the name is just a local variable. Moreover, there is no problem to reference "this" many times inside that quotation; so multiple changes to an element is possible in one go.

In conclusion, I have shown how to use an embedded language interpreted with Javascript to manipulate DOM. 

Beware that it uses Javascript's eval a bit, so you might want not to read content from another sites. The implementation is also slow since the emphasis will be on educational aspects, at least for some time.

The demo can be found here: