//XPath plugin now available via NetBeans plugin portal

The XPath Utility I submitted to the NetBeans Plugin Portal over two months ago has been recently verified against NetBeans 6.7. This makes the plugin now directly available from within the IDE over the Plugin manager (Tools -> Plugins).

One .nbm less to carry with me ;)


//Object Pooling - Determinism vs. Throughput

Object pooling in java is often seen as an anti pattern and/or wasted effort - but there are still valid reasons to think about pooling for certain kind of applications.

The JVM allocates objects much faster from managed heap (young generation; contiguous and defragmented) as you could ever recycle objects from a self written pool running on top of a VM. A good configured garbage collector is also able to delete unused objects fast. GCs in fact don't delete objects explicitly, they rather evacuate all surviving objects and sweep whole memory regions in a very efficient manner and only when its necessary to reduce runtime overhead.

Object allocation (of small objects) on modern JVMs is even so fast that making a copy of immutable objects sometimes outperforms modification of mutable (and often old) objects. JVM languages like scala or clojure make heavy use of this observation. One of the reasons for that anomaly is that generational JVMs are designed to be able to deal with loads of short living objects which makes them inexpensive compared to long living objects in old generations.

Performance does not always mean Throughput

Rendering a game with 60fps might be optimal throughput for a renderer but the performance might be still unacceptable when all frames are rendered in the first half of the second with the second half spent on GC ;). Even if Object Pools may not increase system throughput they can still increase determinism of your application. Here are some observations and tips which might help:

When should I consider Object Pools?

  • GC tuning did not help - you want to try something else
  • The application creates a lot of objects which die in the old generation
  • Your Objects are expensive to create but easy to recycle
  • Determinism, e.g response time (soft real time requirements) is more important for you than throughput

Pro Pooling:

  • pools reduce GC activity in peak times (worst case scenarios)
  • are easy to implement and test (its basically an array ;))
  • are easy to disable (inject a fake pool which returns only new Objects)

Con Pooling:

  • more (old) objects are referenced when a GC kicks in (increases gc overhead)
  • memory leaks (don't forget to reclaim your objects!)
  • cause additional problems in a multi-threaded scenario (new Object() is thread safe!)
  • may decrease throughput
  • cumbersome, repetitive client code

When you decided to use pools you have to make sure to reclaim all objects as soon they are no longer used. One way of doing this is by applying the static factory method pattern for object allocation and a per object dispose method for deallocation.


/**not Thread safe!**/
public class Vector3f {
    
    private static final ObjectPool<Vector3f> pool;
    public float x, y, z;
    private boolean disposed;
    
    static{
        pool = new ObjectPool<Vector3f>(1024);
        for(int i = 0; i  < 1024; i++) {
            pool.reclaim(new Vector3f());
        }
    }

    private Vector3f() {}

    public static Vector3f create(float x, float y, float z) {
        Vector v = pool.isEmpty() ? new Vector() : pool.get();
        v.x = x;
        v.y = y;
        v.z = z;
        v.disposed = false;
        return v;
    }
    
    public void dispose() {
        if(!disposed) {
            disposed = true;
            pool.reclaim(this);
        }
    }
}

To demonstrate the perceived performance difference I captured two flyovers of my old 3d engine. The second flyover was captured with disabled object pools. The terrain engine triangulates the ground dependent on the position and view direction of the observer which makes object allocation hard to predict. The triangulation runs in parallel to the rendering thread which made the pool implementations a bit more complex as the example above.

Every vertex, normal, triangle and quad-tree node is a pooled object (wireframe on mouse over)

on the left: flyover with pre allocated object pools; right: dynamic object allocation (new Object())

Notice the pauses at 7, 17 and 26s on the flyover with disabled pools (right video).

Note on the videos: The quality is very bad since the tool I used created 700MB large files for the 30s videos a lot of frames got skipped. I even sampled them down from 1600x1200 to 1024x768 and limited the fps to 30 but the bottleneck was still the hard disk. This is the main reason why even the left video does not look smooth. (I even had to boot windows the first time in 2 years to use the tool!). I'll try to capture better vids next time.

Conclusion

Using pools requires discipline, is error prone, not good for system throughput and does not play very well with threads. However there are some attempts to make them more usable in case you think you need them. The physics engine JBullet for example uses JStackAlloc to prevent repetitive and cumbersome code by using automatic bytecode instrumentation in the build process. Type Annotations (JSR 308 targeted for OpenJDK 7) in combination with project lombok and/or the automatic resource management proposal might provide further possibilities for simplifying the usage of object pools in java and reduce the risk for memory leaks.


//NetBeans OpenGL Pack 0.5.5 released

NetBeans OpenGL Pack logo The NetBeans 6.7 compatible OpenGL Pack has been updated to version 0.5.5 and is now available on the plugin portal also. The current release is feature compatible with 0.5.4 (release notes) only JOGL and project webstart extensions have been updated to JOGL 1.1.1a security update.

//Using Applets as fallback mode for video on pre html5 browsers

The upcoming html5 standard will make it very easy to embed media of not proprietary formats in webpages. For example video can be embedded in the same way you would probably do it with an image. But what happens when your browser does not support html5 yet?

Firstly: don't panic! Secondly: you could consider using for example the 256kb large cortado applet as fallback mode, since pre html5 browsers will ignore unknown tags like the video tag they will still read the object tag. Using an applet as cross platform fallback mode for playing e.g. theora encoded hd movies is therefore fairly easy - you even don't have to convert the video to an other format.

[Read More]

//Java EE 6 - The Salvation

My Brother Adam Bien released his new book Real World Java EE Patterns - Rethinking Best Practices yesterday. It is available as download or softcover. I am sure you will like it.

You can testread the first two chapters here, more infos are on his blog.

He will check in all samples of his book to this projekt on kenai.com - so make sure you bookmark it or do a mercurial refresh in your favorite IDE when you are interested.


//OpenGL Pack 0.5.4 now ready for NetBeans 6.7

NetBeans OpenGL Pack logoNetBeans OpenGL Pack 0.5.4 is now ready to be used in the upcoming NetBeans 6.7 release, currently as rc2 available.

It wasn't sure if we would be able to ship the GLSL editor in this release since NetBeans 6.7 changed the editor APIs once again. But fortunately the P1 bug was fixed in time and we (and apparently many others, thanks for voting!) can keep using the Generic Language Framework (GLF aka Schlieman) - at least for now since GLF it is now a deprecated/unsupported module.

Build 0.5.4 will break compatibility with NB 6.5. The latest and also all other releases can be downloaded on the project page. I will wait with the upload to the plugin portal until NetBeans 6.7 final is released.

Features/Enhancements:

Anyway. Not much changed since the last release. The most important point is probably that the GLWorker used internally for tasks like shader compilation or capabilities viewer is now more stable on systems which do software rendering (e.g Mesa GL).

I added also an experimental feature which lets you define GLSL shader dependencies similar to java imports.

It is very common in GLSL to reuse code by simple concatenation of files. For example a set of independent shaders can reuse a code fragment defining some generic mathematical functions if the fragment has been concatenated to the top of all shaders which make use of the functions. Editing those kind of shaders would produce compilation errors without a way to inform the editor about those dependencies.

For example the following shader uses the function light() of PerPixelLight.frag by inserting the file ./PerPixelLight.frag at the position of the //import statement.

PerPixelLight.frag


vec4 light(void) {
    // insert fancy light calculation here
}

PlanetShader.frag


//import PerPixelLight.frag

uniform samplerCube cubemap;
varying vec3 vertex;

void main (void) {
    //Cubemap
    gl_FragColor = vec4(textureCube(cubemap, vertex)) * light();
}

When you compile a shader with dependencies you should see something like that in the output window:


All dependencies are listed in the compiler log and even the line numbers of the compiler warnings are translated back to the code fragments, which lets you jump via hyperlink directly to the annotated files.

Just a warning: Please don't define cyclic dependencies, however double imports should work in theory (have I mentioned it is experimental? ;))

Happy coding!


//JavaOne 09 keynote replays and technical session slides available

For those who haven't watched to the keynotes live or per live stream can now watch the replays of all general sessions (on your favourite screen of your live - sorry couldn't resist ;)).

The good thing about that is: you can hit the fast forward button as soon as the marketing guys start talking ;).

The slides of most presentations are also available [updated link] for download.

Especially recommended are James Gosling's toy show (last session), the first two keynotes and of course the pdfs of the technical sessions.


//XPath NetBeans plugin

I built some time ago a NetBeans plugin which simplifies browsing through large xml documents a bit. Just start typing a XPath expression in the text field of the TopComponent and the result of the (last) xml editor in focus are computed and printed in the textarea below. (You can open the XPath window either by using the context menu on xml files or directly via the window menu)

It uses the JDK's javax.xml.xpath package which means XPath 1.0 support.

xpath plugin image

it also supports basic auto completion and localized error messages (in other words: XPathExpressionException.getCause().getLocalizedMessage()).

Github project can be found here. Feel free to use it ;)


//NetBeans OpenGL Pack #2 in most popular plugins ranking

NetBeans OpenGL Pack logoSince I haven't bloged for a long time about the OpenGL Pack, here are some notes for the last two update releases.

Beside updates and fixes the only new feature I added is the OpenGL Quicksearch. It uses the NetBeans 6.5 Quicksearch functionality (similar to ctrl + 3 in eclipse) and searches through several OpenGL resources.



currently are five resources available for search:

power users can restrict the search to a category with the following prefixes ([prefix  space] searchstring  (space searchstring)*):
gl for GL SDK, ext for GL extensions, nv for Nvidia extensions, ati for ATI/AMD Extensions.


JOGL component in NetBeans Formdesigner OpenGL quicksearch GLSL editor

Pictures featuring: Form Designer sample project, GL Quicksearch, updated GLSL 1.3 editor

Changes in 0.5.3:

  • Java Applet and Webstart support
  • OpenGL Quicksearch
  • GLSL editor updated to support GLSL 1.3 (OpenGL 3.0 spec)
  • two new NetBeans Form Designer sample JOGL projects
  • NetBeans 6.5 and JDK6 are the new minimum requirements

Changes in 0.5.2 since last release:

  • JOGL distribution has been updated to version 1.1.1
  • GLSL linker now does not stop linking shaders on compiler warnings
  • the pack should now correctly deploy JOGL and GlueGen's native libraries on all supported platforms (64bit bug)

To download the pack, please visit the plugin portal.

As always, please use the user mailing list for feedback and the issue tracker for bug reports on the project page.

- - - - -

BTW the pack recenty reached the 36k downloads milestone which makes it to the #2 in the most popular plugins category of the plugin portal ;)


//Enabling the new java browser plugin on ubuntu

When you are using Ubuntu and upgraded from older releases to intrepid or jaunty you might have run into a setup bug which causes the browser to keep using the old java plugin despite having latest Java SE and plugin packages installed (e.g 1.6 update 13 from multiverse repository).

To fix this you will have to update some symlinks and let them point to the correct location.

one easy way of doing this is by using the update-alternatives command:

 sudo update-alternatives --all

this will iterate through all symlinks in /etc/alternatives which have more than one alternative and ask you which one to use. Simple update all links which point to:

.../libjavaplugin_oji.so

to the location of the new plugin (e.g for i386):

/usr/lib/jvm/java-6-sun/jre/lib/i386/libnpjp2.so

For all other links just hit Return.

This is a little bit of a brute force approach but there shouldn't be many of them and it is the only way to make sure you don't overlook one of them since they are all called differently ;)

Next time you restart your browser the new plugin should be loaded and applets which use e.g jnlp for deployment (or out of process functionality) should work.


//Java - JavaScript Communication example

Communication between java applets and javascript code is already available since J2SE 1.3 (aka LiveConnect, which was btw. rewritten from scratch in Java 6 update 10 as part of the new plugin) and is really easy to implement. It is a simple way to break out of the sandbox and do things which would usually require full system access (a signed applet + user approval via security dialog). For example applets living in a sandbox are only allowed to read mouse events via the AWT/Swing event mechanism which works as long the mouse is over the applet.

To read e.g the mouse position globaly you would need to call MouseInfo.getPointerInfo().getLocation() which would cause a java.security.AccessControlException: access denied. However, in javascript it is trivial to track mouse events for the whole html document (e.g google adds track onclick x,y events).

All you have to do is to use the object tag instead of applet tag (which is deprecated anyway) and give the object (applet) a name via the id attribute.

<form name="FishForm">   
<object width="256" height="256" type="application/x-java-applet" id="CrazyFish">
<param value="http://people.fh-landshut.de/~mbien/weblog/java_js_interop/launch.jnlp" name="jnlp_href" />
<param value="false" name="draggable" />
</object>
</form>

 now you can simply call methods as usual.

<script language="JavaScript1.2">
//...
document.onmousemove = onMouseMoved;

var tempX = 0;
var tempY = 0;

var applet = document.FishForm.CrazyFish;

function onMouseMoved(e) {
//...
// javascript -> java calls
applet.jsObjectOrigin(findPosX(applet), findPosY(applet));
applet.jsMouseMoved(tempX, tempY)
return true
}
//...
</script>

 the other side is a plain old public method implemented in the java applet.

    /**
* called from javascript.
*/
public void jsMouseMoved(int x, int y) {
//do something usefull
}

 RIA/Web2.0 Observer Pattern in action ;)

(The applet won't work with JRE version < 1.6 update 10 (or the equivalent on Mac OS) since I used the jnlp deployment mechanism, but it wouldn't have been necessary for this particular applet)

... and never forget Web 2.0 is watching you


//There is no place like 127.0.0.1

I have decided to host my blog myself and thought I report about some highlights ;). The reason why I haven't bloged for a long time was actually a configuration issue of the old setup which confused <myDomain>/myBlog with <myBrother'sDomain>/myBlog. Selfhosting is also a good opportunity to play with enterprise software at home in a daily basis (not only the usual try and remove evaluation with constructed dummy projects or brainstormed scenarios).

I initially evaluated OpenSolaris since I was really eager to try out ZFS. But I decided to stay with Ubuntu 9.04 server edition (aka jaunty jackalope) since I already use the desktop version for my workstation. The other reason was the lower memory footprint of Ubuntu (256MB) compared to OpenSolaris (512MB) which is relevant if you run your site on over 10 years old hardware like I do.

Ubuntu has great software repositories which just provide everything to get started (e.g Sun JDK6.13 and in theory also glassfish, netbeans, eclipse, your favourite database etc but i prefer in those cases zip files ;-) ).

Installation with gimmicks like Software RAID (RAID 1), firewall configuration went flawlessly. The combination GlassFish2.1 + Roller 4.0.1 + hsql 1.8 worked also on first attempt.

a few thing I noticed:

  • 5 years ago the whole process would take far longer than two evenings
  • -XX:OnOutOfMemoryError=<send me a mail command> (since java 1.6) is really cool
  • Cloudcomputing is overrated (web services should always be weather independent)
  • There is no place like 127.0.0.1

don't worry the upcomming entries should be more technical - this is mainly a test ;-)


//FishFarm wins second prize in GlassFish Community Innovation Awards Program

I recently won with FishFarm the second prize in the GlassFish Community Innovation Awards Program -  which is pretty cool. I would never have thought that I have a chance to win something in the GAP.

Since you probably don't know what FishFarm actually is, I will try to introduce it with this entry.

Formal:

FishFarm = Shoal + Fork/Join Framework
Shoal = simple to use clustering framework currently based on JXTA and used within GlassFish

Fork/Join Framework = pretty cool concurrency framework for local parallelization of tasks (jsr166y targeted for Java 7)

=> FishFarm = simple but pretty cool solution for distributing concurrent tasks over a p2p network [q.e.d.]

Informal:

Project FishFarm is a simple solution for distributing computational intensive tasks over the network based on Java SE APIs.

The goal of this project is to take any task written in the Fork/Join Framework (JSR166y targeted for Java 7) and distribute the computation over multiple nodes in a grid. FishFarm introduces no new frameworks and is also no full featured distribution system.

The initial focus was to make the ForkJoinPool (which is a core part of jsr166y) distributable with as few code changes as possible. Thanks to Doug Lee these modifications are now in trunk of his Fork/Join Framework and he even provided a handful of utility methods to make further extensions simpler.

How it works:

All you need to make your Application distributable is to replace ForkJoinPool with FishFarm's DistributedForkJoin pool.


  ForkJoinPool pool = new DistributedForkJoinPool();

  // submit as many tasks you want (nothing changed)
  Future futureResult1 = pool.submit(new MyTask());
  Future futureResult2 = pool.submit(new LongRunningTask());

  // block until done
  System.out.println("result of task 1: " + futureResutl1.get());
  // or ask if done
  System.out.println("task 2 isDone=" + futureResutl2.isDone());

Every DistributedForkJoinPool is member of a peer2peer network and automatically steals work from overstrained pools if idle. DistributedForkJoinPool extends ForkJoinPool and will complete submitted tasks even when working offline or on node failures. No additional configuration required.

I wouldn't call it ready for production yet but it should be stable enough to have fun ;-)

webstartable demos


 -------

In case you are wondering why you are reading this entry via the RSS feed of my brother's (Adam) blog. This is a bug which confuses both urls, I hope this should be fixed with the next roller deployment.

This is Michael - over and out ;-)


//Web 3.0 alias Java 6 update N + JavaFX Desktop profile anounced

"The Workaround" - better known as Web 2.0 is now no longer necessary ;).

With Java 6 update 10 and later + JavaFX Desktop profile it is possible to run REAL applications (lets call them applets) out of process in the browser. The browser sandbox is now optional, if you like the applet to persist, just drag it out of the browser and you can use it stand alone (same process -> same state).

If you close it, it simple moves back to the browser, but what happens if you close the browser when the stand alone applet is still up and running you may ask? Now you technically transformed your applet to a webstart application and installed it on your system. Isn't that cool?!! Installation has never been so easy.

Additional to that a great demonstration of the Adobe - Photoshop/Illustrator exporter plugin has been shown on the JavaOne tech session where a professional designer and a java geek developed side by side a cool looking animated application without knowing concrete implementation/design details from each other*. Now you are able not just to develop techical superior applications you can also make them look awesome with minimal effort. (technically the exporter plugin exports each layer so coolest things will be possible)

But this is not everything: missing audio and video codecs are now ready to use, BDLive which brings network to your Blu-ray player has been shown and BDLive developer discs are now availabe at BDLive.com.

The early access JavaFX SDK will be available in June. More information on javafx.com (well the webside does not look and feel very good but thats probably because its not written in JavaFX script... ;) )

-------

*Sun Tech Session 3

Applets Reloaded with Ken Russell

Livedemo including sourcecode and documentation of the draggable applet feature is now available.


//NetBeans OpenGL Pack 0.5.1 released

The NetBeans OpenGL Pack 0.5.1 has been released. This is a small bugfix release with no major features added.

Changes since last release:

  • updated JOGL distribution to v1.1.1rc8
  • updated JOGL demos (added gears Applet demo)
  • added JOGL API support module for doc and source integration
  • webstart support for JOGL project templates (just enable it in the project properties)
  • improved JOGL deployment code (fixed some issues on 64bit systems)
  • bug fixes

To download the pack please visit the plugin portal page.

The OpenGL pack will be soon available via plugin portal update center (Tools | Plugins). This will make our old  "NetBeans OpenGL Pack update center" obsolete (and improve the download speed!). However, we will use the old update center for experimental not final releases in future.

As always, please use the user mailing list for feedback and the issue tracker for bug reports on the project page.

-------

Write once run everywhere [1][2][3][4][5][6] - seems like the platform agnostic JOGL deployment is working ;)