Category Archives: Developers log

I have to come up with some names for the different technologies soon. Smart now have two terms for compiler and also two levels of RTL: the low level RTL that is a part of Eric’s compiler, and a high level RTL which is our “VCL” clone. Im thinking of naming our component library the VJL (visual javascript library), and instead of refering to the compiler infrastructure as “the compiler”, it is more fitting to call it “the build tool”.

One vs. many

As the test group knows, the build-tool takes your source-code, invoked the compiler and finally glues everything together into a final, solid HTML5 document. Since our system was originally designed for iOS development only, and since Smart components (including forms, which is our version of pages in a way) are purely javascript based – a fully operational “app” really only needs one HTML file.

In Alpha 2 i removed the injection of code directly into the document.body section and introduced “Application.Display.View” to the VJL. The reason for this was make the application regard “the screen” abstractly. So by altering what “display.view” points to – you can make your apps live inside any DIV you wish.

A couple of testers have reported back that it could be handy to separate the RTL from the program code. There are a number of scenarios where you might, for example, want to run multiple Smart apps on the same document. If you run a website and want to get rid of Flash for instance, it would be nice to code the new banner in Smart and also host a game or a cool menu written in Smart as well.

Having talked it over with Eric it became clear that this should be reasonable to achieve, but it will come at a price: namely that it defeats the latest compiler switch named “smart linking”. When you smart link your apps, only the code you have used is included in the final binary (in our case “binary” is javascript). It’s the same technique Delphi and C++ uses to make programs as small as possible.

So if we include the feature then it will disable the smart linking option. I am experimenting with this now, by dumping the entire RTL into a separate file. If it doesn’t produce any negative side effects (except a small speed penalty and some bloat) then I’ll include it as an option. Eric might slap me over the head for playing around with this – but it would (in my view) complete the parallel of “Delphi” for javascript.

Reusable libraries

One of my long term goals is to add a new project type to Smart for writing libraries. Which would be the javascript incarnation of DLL’s. Since we have our own IDE we could generate a class-map for the library – which you could then import later so the Smart IDE can keep track of what the libraries export. A bit like a Windows TLB file when working with COM objects.

We should be able to poll this off by isolating the compiled code in it’s own name-space. It would then have it’s own RTL and library – but due to the nature of Javascript, it should actually work 🙂 It would mean a little tweak to the RTL but we are constantly tweaking anyhow.

Why are libraries important? Components, that’s why. By pre-compiling and obfuscating your code – you could write and sell your own components. The other benefits are self evident I think.

For the past week and a half things have been quite calm on our website, this is not due to lack of work but rather focusing on some of the more tricky aspects of the IDE. So here they are in no particular order:

Alternative rendering

As an alternative to Chromium I attempted to wrap the Opera DLL’s last weekend, but the differences between Safari and Opera (both style and javascript wise) were to large – so I decided to drop it. Chrome is the browser most compatible with Safari (they both use the same webkit foundation) and as such we will stick with TChromium. We also had to backtrack to find the most stable version of TChromium since the latest revisions simply don’t work as expected.

Keyword proposal

Keyword proposal now works as expected

This is an issue that has been driving me nuts for a while, but finally I tracked down the bug (wrong cache pointer .. sigh) – and Smart Mobile Studio now have keyword proposal in the box. We will naturally aim to expand this into the realm of code-completion and other advanced features in the future.

XML file format

In the first versions of the Alpha release we used a custom binary file format. This file format is now replaced by a proper XML format, making it easier to expand the system with new options and data without breaking backwards compatibility. You will need to copy and paste out your units to make the transition though, but it’s a small price to pay for a more versatile system. The new file format also supports external unit files (much like Delphi has by default) and we added a recursive search method to locate implicitly referenced units. Also, a new folder as been added to the mix: “libraries”. If you have made some utility units place those files under IDE/Libraries and the IDE will automatically find them.

What’s next

Once we have completed some of the tickets on fogbugz the final battle begins, namely offscreen rendering of components and the visual form designer. Then we brush up on the look & feel of the IDE and give the standard components more features and bling — then it’s release time.

Crack open the Redbull, im ready to rock 🙂

Using variants as mediators is now possible under Smart Mobile Studio. This removes some of the burden when interacting with native javascript objects or external libraries:

[sourcecode language=”delphi”]
procedure TMyObject.InvokeByVariant;
var
mSrc: variant;
Begin
//get the browsers window object
asm
@mSrc = window;
end;

//invoke our method
handleCallback(mSrc);
end;

procedure TMyObject.HandleCallback(a:variant);
Begin
// treat "a" as an object
a.alert(‘this is a test’);
end;
[/sourcecode]

Note: Variants are treated as “unknown” by the codegen. It has no compiler operators for it, meaning you cant write “if a<>1 then” or “a:=12”. Variants are primarily used to transport native values and objects between methods dealing with javascript objects or browser objects.

In the above example we simply calls a procedure passing the native browser object “window” as a parameter. In the HandleCallBack method we invoke the alert method of that object. Once again, Smart deals with variants as unknown entities – so there is no checking that “alert” actually exists.

The collective wisdom of the WC3 can be questioned from time to time.

One of the trickiest and most frustrating aspects of javascript coding has to be the async file-system. Where opening a file and looping through a read sequence sounds easy enough (and is in most languages) – being async means that everything is handled through events. Why you say? Because javascript is by nature single threaded, so the cpu time has to be divided between all the action that takes place inside the browser. And that includes accessing files and folders.

In short, when you open the filesystem – the actual object you get back is delivered through an event. When you want to read X number of bytes from a file, you invoke it in one place and handle the actual reading someplace else. So it’s not exactly linear and object pascal friendly to say the least.

Using the force, Sith style

To solve this I have come up with the concept of disk actions. A disk action in this case is basically an object that handles everything associated with the action you want to execute (loadfile, savefile, copy etc.) – this gives us the benefit of isolating the code that invokes the filesystem and it’s event handler(s) in one place. And those that want to access the filesystem through Smart Mobile Studio wont have to deal with the ridicules visions of the World Wide Web Consortium (WC3).

My wrapper aims at handling things like this:

[sourcecode language=”delphi”]
FFileSystem:=TW3FileSystem.Create;
try
FFileSystem.open(5 * CNT_MEGABYTE,fstPersistent);
try
FFileSystem.Execute(TW3OpenFileAction.Create
(‘prefs.ini’,IniFileRead);
finally
FFileSystem.close;
end;
finally
FFileSystem.free;
end;

Procedure IniFileRead(sender:TObject;aData:TW3Buffer;
aFilename:String);
Begin
//File is read into a buffer (actually a "blob"
//Use the methods of TW3Buffer and TW3View(ers) to
//access the data in "reader" style
w3_showmessage(aData.toString);
end;
[/sourcecode]

Always been more of an empire man

But due to the nature of the javascript callback system, we probably have to move several aspects of the above into events. But at least I will isolate the events in one object, making it a hell of a lot easier to work with than native javascript.

For example, “FFileSystem.Execute” wont be a legal call in the above example, because it will be executed before the event which delivers the file-system native object has called us back. To solve this I have some tricks up my sleeve – but I think it’s wise to play it safe when it comes to building a new RTL from scratch. So expect some event management.

The answere is 42

I also built in a fail-safe for error codes. As I was testing the same code in both Chrome, Safari and FireFox – i noticed that the error codes where different between two browsers (which shall remain anonymous but involves a fox and some iron). The error names were the same (e.g: FileError.QUOTA_EXCEEDED_ERR) but the integer values could differ.

So instead of just hard-coding the numbers I used the newly added TW3Dictionary object(s) to map the error codes in the browser to it’s Object Pascal type enumeration. So even if the browser alters the actual error code our code will work just fine.

[sourcecode language=”delphi”]
//Register known errors in the constructor
FErrors:=TW3IntDictionary.Create;
registerError(fseQuotaExceeded);
registerError(fseNotFound);
registerError(fseSecurity);
registerError(fseInvalidModification);
registerError(fseInvalidState);
registerError(fseUnknown);

//Do evil things with butter
Procedure TW3FileSystem.RegisterError(aKind:TW3FileSystemError);
var
mKey: String;
Begin
if supported then
Begin
try
mKey:=IntToStr(ErrorToNative(aKind));
FErrors.Values[mKey]:=ord(aKind);
except
on e: exception do
Raise EW3Exception.CreateFmt(CNT_ERR_METHOD,
[‘RegisterError’,ClassName,e.message]);
end;
end;
end;
[/sourcecode]

Thank god I only have the audio and video objects left to wrap, because I’m getting a bit tired of low level javascript. But the foundation is getting tweaked and polished (Primoz Gabrijelcic have really given it a run for it’s money, testing every avenue and tidbit in the framework. I dont think I have ever seen a person consume and adapt to information that fast. Also the wizard of code which is Eric Grange is ever watchfull and actually knocked out a full game in under a day after release), so when the foundation is done for version 1.0 –  I can fully dedicate myself to high-level component writing.

Having said that — there is plenty of room for you guys to create your own units and start creating components for Smart Mobile Studio!

So far people have created games, advanced mandelbrot explorers, particle systems, hacker group demos and even classes that connect directly to RemObjects services (!) – and we aren’t even out of the alpha phase. Imagine what version 2.0 will look like. Buckle up dorothy because we’re just getting started (!)

Oh almost forgot.. Whenever I have the spare time, I will be implementing a converter for .mod music files (sound-tracker, pro-tracker: those old sample based music files). Im not sure the JS timer system is up for the challenge, perhaps we have to use web-workers (threads) to handle the dispatching – but I really want to add the frosting on the cake later on. No one has ever done anything like this in Javascript – so why not send a spartan to do it 😉

Fair play

And please. Try to avoid copying software. We are a two man company and have dedicated a lot of resources to polling this off. If you use it in some way – please support our work and buy a license when it’s released. If you want your friend to try it out then just ask us and he can register with the alpha team. We hear code written in Smart is already being used in production scenarios, and it’s sort of self-defeating for the community if what we hear is true.

Our prices will be very reasonable. Somewhere between $200 – $400 for the first version which is very humble concidering 6+ months of work has gone into research, figuring out how to do things, building a model and bridging technologies.

Smart Mobile Studio is becoming a weapon of mass construction – ok so let’s show those javascript punters what were made of 🙂

A sprite sheet is basically just an image. But it’s an image that is organized in a special way, usually with a 32×32 or 64×64 pixel grid where each cell contains either a character animation frame – or a background tile. Back in the old Amiga days of 8 bit graphics and hand crafted sprites this was the only way to go.

Since the world of mobile javascript game development is more or less en-par with the old Amiga (in terms of cpu power and restrictions. Which is amazing since the Amiga had a 1hz motorola mc68000 CPU) an example of a spritesheet class might come in handy.

Classical sprite sheet from sega

A simple spritesheet class

TW3SpriteSheet = class(TObject)
private
  FImage:     TW3Image;
  FReady:     Boolean;
  FColumns:   Integer;
  FRows:      Integer;
  FTotal:     Integer;
  FWidth:     Integer;
  FHeight:    Integer;
  FOnReady:   TNotifyEvent;
  procedure   HandleImageReady(sender:TObject);
  procedure   SetWidth(aValue:Integer);
  procedure   Setheight(aValue:Integer);
  Procedure   ReCalcInfo;
public
  Property    OnReady:TNotifyEvent read FOnReady write FOnReady;
  Property    SpriteWidth:Integer read FWidth write setWidth;
  Property    SpriteHeight:Integer read FHeight write setHeight;
  Property    Ready:Boolean read FReady;
  Procedure   LoadImage(aUrl:String);
  Procedure   Draw(Canvas:TW3Canvas;dx,dy:Integer;FrameIndex:Integer);
  Constructor Create;virtual;
  destructor  Destroy;Override;
end;

constructor TW3SpriteSheet.Create;
begin
  inherited Create;

  FWidth  := 32;
  FHeight := 32;
  FColumns := 0;
  FRows  := 0;
  FTotal := 0;

  FImage := TW3Image.Create(NIL);
  FImage.OnLoad := HandleImageReady;
end;

destructor TW3SpriteSheet.Destroy;
begin
  FImage.onLoad:=NIL;
  FImage.free;
  inherited;
end;

procedure TW3SpriteSheet.setWidth(aValue:Integer);
begin
  FWidth:=w3_ensureRange(aValue,4,MAXINT);
  ReCalcInfo;
end;

procedure TW3SpriteSheet.setheight(aValue:Integer);
begin
  FHeight:=w3_ensureRange(aValue,4,MAXINT);
  ReCalcInfo;
end;

procedure TW3SpriteSheet.ReCalcInfo;
begin
  if FReady then
  begin
    FColumns:=FImage.Width div FWidth;
    FRows:=FImage.Height div FHeight;
    FTotal:=FColumns * FRows;
  end else
  begin
    FColumns:=0;
    FRows:=0;
    FTotal:=0;
  end;
end;

procedure TW3SpriteSheet.HandleImageReady(sender:TObject);
begin
  FReady:=True;
  ReCalcInfo;

  if assigned(FOnReady) then
    FOnReady(self);
end;

procedure TW3SpriteSheet.LoadImage(aUrl:String);
begin
  FReady:=False;
  ReCalcInfo;
  FImage.LoadFromUrl(aUrl);
end;

procedure TW3SpriteSheet.Draw(Canvas:TW3Canvas; dx,dy:Integer; FrameIndex:Integer);
var
  sx,sy:  Integer;
begin
  if  FReady
  and assigned(canvas)
  and (FrameIndex>=0)
  and (FrameIndex<FTotal) then
  Begin
    sx:=FrameIndex mod FColumns;
    sy:=FrameIndex div FRows;
    sx:=sx * FWidth;
    sy:=sy * FHeight;
    canvas.DrawImageEx2F(FImage.tagRef,sx,sy,FWidth,
    FHeight,dx,dy,FWidth,FHeight);
  end;
end;

Using the class

Using the class is simple. Initialize it in TApplication.ApplicationStarting method and load in a picture, like this:

  FSheet := TW3SpriteSheet.Create;
  FSheet.LoadImage('res/sprites.png');

When the actual sprite-sheet image is loaded, the “ready” property will be set to true. So in your PaintView method always check the ready flag before accessing it:

  if FSheet.Ready then
  begin
    FSheet.Draw(Canvas,100,100,10); // draw tile #10
    FSheet.Draw(Canvas,133,100,11); // draw tile #11
    FSheet.Draw(Canvas,166,100,12); // draw tile #12
  end;

Enjoy!

We have received some questions regarding the w3inet.pas unit that ships with the alpha release. Most of the questions is in the line of “why doesn’t this code work” when trying to connect to a server.

The reason you cannot connect to an external host, especially one outside your own domain, is actually a rule imposed on us by the W3C standard (see http://www.w3.org/TR/XMLHttpRequest/ for the documentation). This limitation is called the “same origin” policy, where a website is only allowed to access data and information from the domain it came from. It was this rule that put an effective stop to people stealing content (that was so common in the early days of the Internet).

In other words, you can only use it to access files from your own domain. But, since you are developing on a local machine and not your server this means we have to localize our operations a bit:

1. Place the file (an XML file for instance) you want to access in your res or lib folder (add it to your project as a resource)
2. use relative paths when accessing it (instead of “http://domain/path/name” use “/res/mystuff.xml”)
3. As an alternative, edit your windows host file and point www.yourdomain.com to localhost

Accessing web services

The easiest way to bypass the origin rule is to place your development machine in the domain you want to target, or simply add a line to your host file (we used to do this with our WebSnap services all the time). If these two options are not possible, simply install a public domain proxy on your development machine so that you can control the source.

Future development

Since we have a built in server in our IDE – this issue will be taken care of. We will have a dialog that allows you to define an external service – and then access that as a call to the local server. But for now, the rules of the game is the same as any JavaScript developer have to deal with.

OP4JS about to go Alpha

I posted a picture of the OP4JS pre-Alpha testbed to my Facebook group, and a member found it interesting that it had an ASM section in the code. How can there be an ASM section in a system that targets JavaScript you wonder?

Well, assembler for OP4JS is basically what assembler would be to a real Delphi application. In this case: raw JavaScript. Eric Grange, my friend and author of our compiler, has done a great job on transposing classical Object Pascal terminology onto the (let’s face it) bewildering typology of JavaScript.

Take this simple procedure for instance:

procedure TMyObject.Testing;
var
  mRef: TObject;
begin
  mRef := tagRef;
  asm
    (@mRef).style.width=100 + "px";
  end;
end;

What we do here is to first get a handle to the our tag (the tagRef is a public property of TW3TagObject, the class TW3CustomControl inherits from). We then enter the ASM section, use that reference to access and alter the width style property. Ofcourse you dont have to deal with this stuff unless you want to, because OP4JS will ship with a rich set of controls ready to be used. And rolling your own controls is so simple it’s almost ridicules.

The squiglies

But why the @ prefix you wonder? Well, when we obfuscate our code (make it unreadable, or less “stealable” for the average hacker), all the names of variables and methods are altered. As such, the compiler needs to be able to quickly figure out if you are using a JS based variable or an Object Pascal variable. So all Object Pascal variables have to be prefixed with the @ character. Due to the nature of Javascript you also need to isolate direct references to object fields and native fields inside ( [ .. ] ) sections, like in this example:

asm
  (@self.FWidth) = parseInt( (@mRef).style.width );
end;

Where did my pointers go?

JavaScript don’t have pointers, at least not in the proper sense of addresses and memory buffers. What it does have are object references and data references (it also has a weak reference counted garbage collector). So under OP4JS we dont have a type called “pointer” because it would screw up the reality of compiling to JavaScript. Instead, TObject serves the function of a universal object reference, and variant serves as a universal data reference. Variants are special however, because we dont have any operators dealing with it directly (so you cant write: A:=1 + someVariant). The variant type is just used to transport data from one method to another.

But, all the standard types are there: string, boolean, float, integer and so on. You also have records, interfaces and enumerations to play with. It really is a neat system once you get the hang of it.

  function getSomeData: Variant;
  begin
    asm
      /* return a data structure */
      @result = {first:"hello",second:"hello2"}
    end;
  end;

  procedure handleSomeData(aValue: Variant);
  begin
    asm
      /* access the data structure passed in */
      alert(@aValue.first); //will show "hello"
    end;
  end;

Super simple custom control

To demonstrate how extremely simple it is to create your own controls under OP4JS, let’s make a scroll-text. As you probably know HTML have a tag called marquee that scrolls text in a variety of ways. It has been called an evil tag because it consumes a lot of CPU speed (and that part is very true), but just for the hell of it – let’s make a super-simple OP4JS version of it:

unit myscroller;

interface

uses w3system, w3components;

type

TW3Marquee = Class(TW3CustomControl)
private
  FText:    String;
  procedure SetText(aValue:String);
public
  Property  Text:String read FText write SetText;
End;

implementation

procedure TW3Marquee.SetText(aValue:String);
Begin
  //Keep the text
  Ftext:=aValue;

  // Cheat and inject the tag into our innerHTML
  // This works because TCustomControl has already created
  // a tag for us. So we just inject the content
  innerHtml:='<marquee style="'
  + 'font-family: verdana;'
  + 'width: 100%;'
  + 'height: 100%;'
  + '-webkit-marquee-increment:3px;'
  + '-webkit-marquee-speed:fast;'
  + '-webkit-marquee-direction:backwards;'
  + '-webkit-marquee-repetition:infinite;'
  + '">' + FText + '</marquee>';
end;

end.

Naturally this is just a quick example. A fully functional and quality control would have a slightly different layout. There we would isolate the marquee in it’s own tag object. But the approach is perfectly valid, although a bit slap-dash 🙂

OP4JS does not work by injecting text into the DOM, that is to slow. We create tag objects via the Javascript API and access our objects by reference. This is why OP4JS apps are much faster than those produced by other JS compilers. We try to avoid the use of getElementById() because it results in poor performance.