Michael Bien's Weblog

Since I am doing a lot with OpenCL lately I decided to try to improve the tooling around OpenCL a bit. A weekend later the NetBeans OpenCL Pack was born :).

Features Including:

• OpenCL Editor with syntax highlighting code completion and CL reference pages integration
• OpenCL compiler integration
• In-editor annotations of compiler warnings and errors updated as you type
• JOCL project template

Technical Details:

The editor uses ANTLR as parser and lexer. This allows such simple things like keyword highlighting and also more complex features like semantic highlighting, formatting and auto completion (formatting is not yet implemented). It can also detect and report syntax errors, however this feature is automatically disabled if an OpenCL compiler is present on the host system. All with help of JOCL detected OpenCL implementations can be used as compiler backend.

Instead of using the old OpenGL Pack as template I decided to write it from scratch using latest NetBeans 7 and Java 7 APIs. So you will have to start NB with JDK7 to be able to use it.

Download

you can download it from the NetBeans plugin portal [mirror], sourcecode is on github

feedback and/or contributions/bugreports are as always appreciated

Screenshots:

have fun!

You might remember the project called Jackpot which came out of SunLabs and had James Gosling involved with. It was basically a way to declaratively define refactoring rules, allowing for example, to migrate client code between incompatible third party libraries. The project has been integrated into NetBeans as the IDE's refactoring engine since then. NetBeans 6.9 uses Jackpot for most of the in-code hints and code inspections for instance.

There were various ways to specify the transformation rules, e.g. via a special declarative language or even in Annotations directly in the library-code which would cause incompatibilities (or e.g in conjunction with @Deprecated).

Jan Lahoda recently started with the efforts to make the project usable as standalone tool again. Jackpot 3.0 is available via bitbucket for early adopters.

Back to the Future

I used this opportunity to test jackpotc (the jackpot compiler) with JOGL. What I tired is to provide transformations which transform old JOGL 1.1.1 code into latest JOGL 2 compatible client code. So firstly thanks to Jan for fixing all the bugs we ran into while testing the experimental command line compiler.

The first thing I did was to transform the code to properly use OpenGL profiles. As testcode i will use the famous Gears OpenGL demo as example (but those kind of automatic transformations will only pay of if you use them on large codebases). Since it was written against JOGL 1.1.1 it can only use OpenGL up to version 2.x, which means we can simply use the GL2 profile.

Transformation Rules

'JOGL2 API change: javax.media.opengl.GL -> javax.media.opengl.GL2':
javax.media.opengl.GL=>javax.media.opengl.GL2;;

'JOGL2 API change: new javax.media.opengl.GLCapabilities(javax.media.opengl.GLProfile)':
new javax.media.opengl.GLCapabilities()=>
new javax.media.opengl.GLCapabilities(javax.media.opengl.GLProfile.get(javax.media.opengl.GLProfile.GL2));;

'JOGL2 API change: GL gl = drawable.getGL() -> GL2 gl = drawable.getGL().getGL2()':
$d.getGL() :: $d instanceof javax.media.opengl.GLAutoDrawable=>
$d.getGL().getGL2();; 

Just by looking at the transformation rules you can easily see that it is far more powerful than any simple text replacement could be. Jackpot uses javac and can therefore work with full qualified names, instanceof and more. It will also correctly fix imports for you (there is currently a open bug in this area). The quotes are used as description string which will be printed when jackpotc runs on every code occurrence which applies to that rule.

Invoking Jackpot

jackpotc -sourcepath $SRC -cp $LIBS -d $OUTPUT\
         -Ajackpot30_extra_hints=./jogl1Tojogl2.hint $FILESET

$LIBS must contain both library versions, JOGL 1.1.1 and JOGL 2. This is not optimal but it will probably work in most situations to just use both without thinking about an particular ordering or the need to do multiple iterations.

Results

./test/oldgears/src/jogl111/gears/Gears.java:54: warning: JOGL2 API change: GL gl = drawable.getGL() -> GL2 gl = drawable.getGL().getGL2()
    GL gl = drawable.getGL();
...

@@ -51,7 +51,7 @@
     // Use debug pipeline
     // drawable.setGL(new DebugGL(drawable.getGL()));
 
-    GL gl = drawable.getGL();
+    GL2 gl = drawable.getGL().getGL2();
...

You can find the complete demo and all ready-to-run shellscripts in the tools/jackpotc folder inside JOGL's git repository. The classic JOGL 2 Gears demo can be found in form of an applet here (uses latest hudson builds... can be unstable).

happy coding!

If you are using GIT as SCM and NetBeans as IDE you should probably check out NBGit. The plugin integrates GIT in NetBeans in the same way as the out of the box Mercurial support does it. In fact both modules have the same origin since nbgit is a fork of the mercurial integration project and incrementally adds features to catch up.

NBGit Version 0.3 is already fairly stable and provides the basic set of features you would expect from distributed versioning system IDE integration.

Features

• Graph visualization of parallel branches (Browser similar to giggle)
• Versioning History (git log)
• Show changes (git status)
• update/commit/reset
• clone/clone other/git init
• custom actions (custom git commands)
• diff
• in-editor annotation of code changes
• ignore files (parsing '.gitignore' files)
• git properties (username, email etc via options)

The project is developed by volunteers outside Sun, if you like to see GIT integration as out-of-the-box feature in a future version of NetBeans please vote for this RFE.

I use the plugin for most of my open source projects and haven't experience any serious issues so far. I would say its already safe to use since you can't do anything wrong if you do a 'git status' -> 'git push' via command line as last step anyway.

June 16 2010, updated blogpost: OpenGL 4

JOGL 2 supports several OpenGL Profiles. In this blog entry I try to explain what profiles are and why they are needed.

History

SGI released the first OpenGL specification 1992. Since this point OpenGL 1.x constantly evolved (under the ARB and later Khronos Group) by adding new functions to the core API. This went well until programmable graphics hardware became mainstream and shaders became suddenly more flexible and efficient as the generic fixed function pipeline.

OpenGL 2.x was the last version in which you could freely mix the fixed function pipeline with the programmable pipeline (as a core feature).

With the release of OpenGL 3.0 the whole fixed function pipeline has been deprecated but you could still use it if you haven't requested a forward compatible context.

OpenGL 3.1 and 3.2 removed most deprecated functionality from core specification, however some implementations (e.g. Nvidia drivers) still allow to get them back via an optional compatibility extension. Since 3.1 was the first release which broke compatibility, it is often seen as major OpenGL 3 release.

JOGL 2 (JSR 231)

JOGL 1.1.1 lived in the timeframe up to OpenGL 3.0 which made it easy to stay in sync with the spec. To be able to solve the issue with the deprecation of functionality, JOGL 2 (JSR maintenance release) introduces an abstraction of the original OpenGL versioning called Profile. Profiles allow Java applications to be written in a way which allows compatibility with multiple OpenGL versions at the same time. Since OpenGL ES (GL for embedded systems) has overlapping functionality with OpenGL itself it opened the opportunity to add even Profiles which bridge desktop and embedded implementations. The class diagram below shows the dependencies between all available Profiles.

Before you start writing a JOGL application you will have to decide first which GLProfile you want to use. The code snippet below lists all currently supported profiles (extracted from GLProfile).

Current list of supported profiles and their mapping to the implementation versions

    /** The desktop OpenGL compatibility profile 4.x, with x >= 0, ie GL2 plus GL4.

        bc stands for backward compatibility. */
    public static final String GL4bc = "GL4bc";

    /** The desktop OpenGL core profile 4.x, with x >= 0 */
    public static final String GL4   = "GL4";

    /** The desktop OpenGL compatibility profile 3.x, with x >= 1, ie GL2 plus GL3.

        bc stands for backward compatibility. */
    public static final String GL3bc = "GL3bc";

    /** The desktop OpenGL core profile 3.x, with x >= 1 */
    public static final String GL3   = "GL3";

    /** The desktop OpenGL profile 1.x up to 3.0 */
    public static final String GL2   = "GL2";

    /** The embedded OpenGL profile ES 1.x, with x >= 0 */
    public static final String GLES1 = "GLES1";

    /** The embedded OpenGL profile ES 2.x, with x >= 0 */
    public static final String GLES2 = "GLES2";

    /** The intersection of the desktop GL2 and embedded ES1 profile */
    public static final String GL2ES1 = "GL2ES1";

    /** The intersection of the desktop GL3, GL2 and embedded ES2 profile */
    public static final String GL2ES2 = "GL2ES2";

    /** The intersection of the desktop GL3 and GL2 profile */
    public static final String GL2GL3 = "GL2GL3";

Note: GL2 Profile supports OpenGL up to version 3.0 (included) - this is not a bug: OpenGL 3.1 was the big game changer

The next two code snippets show the basic steps how to set up OpenGL with JOGL 2.

Context creation

        //create a profile, in this case OpenGL 3.1 or later
        GLProfile profile = GLProfile.get(GLProfile.GL3);
        
        //configure context
        GLCapabilities capabilities = new GLCapabilities(profile);
        capabilities.setNumSamples(2); // enable anti aliasing - just as a example
        capabilities.setSampleBuffers(true);
        
        //initialize a GLDrawable of your choice
        GLCanvas canvas = new GLCanvas(capabilities);

        //register GLEventListener
        canvas.addGLEventListener(...);
        //... (start rendering thread -> start rendering...)

Rendering

    public void display(GLAutoDrawable drawable) {
        GL3 gl = drawable.getGL().getGL3();
        gl.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT);
        //.. render something
    }

Summary

Profiles make JOGL 2 very flexible and allow it to build modular and portable applications. For instance part A of an application can be written against the GL2ES2 interface and part B (which is more hardware specific) against the GL3 interface. This would in theory allow to reuse A in an embedded application and B could e.g. disable itself on old desktop hardware which runs only OpenGL 2.x or fall back to a GL2 implementation.

More information can be found on JogAmp.org (direct link to javadoc)

The next release of the OpenGL Pack for NetBeans will fully support JOGL 2. Beta builds can be found here (builds contain JOGL2 beta5):

The XPath Utility I submitted to the NetBeans Plugin Portal over two months ago has been recently verified against NetBeans 6.7. This makes the plugin now directly available from within the IDE over the Plugin manager (Tools -> Plugins).

One .nbm less to carry with me ;)

NetBeans OpenGL Pack 0.5.4 is now ready to be used in the upcoming NetBeans 6.7 release, currently as rc2 available.

It wasn't sure if we would be able to ship the GLSL editor in this release since NetBeans 6.7 changed the editor APIs once again. But fortunately the P1 bug was fixed in time and we (and apparently many others, thanks for voting!) can keep using the Generic Language Framework (GLF aka Schlieman) - at least for now since GLF it is now a deprecated/unsupported module.

Build 0.5.4 will break compatibility with NB 6.5. The latest and also all other releases can be downloaded on the project page. I will wait with the upload to the plugin portal until NetBeans 6.7 final is released.

Features/Enhancements:

Anyway. Not much changed since the last release. The most important point is probably that the GLWorker used internally for tasks like shader compilation or capabilities viewer is now more stable on systems which do software rendering (e.g Mesa GL).

I added also an experimental feature which lets you define GLSL shader dependencies similar to java imports.

It is very common in GLSL to reuse code by simple concatenation of files. For example a set of independent shaders can reuse a code fragment defining some generic mathematical functions if the fragment has been concatenated to the top of all shaders which make use of the functions. Editing those kind of shaders would produce compilation errors without a way to inform the editor about those dependencies.

For example the following shader uses the function light() of PerPixelLight.frag by inserting the file ./PerPixelLight.frag at the position of the //import statement.

PerPixelLight.frag

vec4 light(void) {
    // insert fancy light calculation here
}

PlanetShader.frag

//import PerPixelLight.frag

uniform samplerCube cubemap;
varying vec3 vertex;

void main (void) {
    //Cubemap
    gl_FragColor = vec4(textureCube(cubemap, vertex)) * light();
}

When you compile a shader with dependencies you should see something like that in the output window:


All dependencies are listed in the compiler log and even the line numbers of the compiler warnings are translated back to the code fragments, which lets you jump via hyperlink directly to the annotated files.

Just a warning: Please don't define cyclic dependencies, however double imports should work in theory (have I mentioned it is experimental? ;))

Happy coding!

I built some time ago a NetBeans plugin which simplifies browsing through large xml documents a bit. Just start typing a XPath expression in the text field of the TopComponent and the result of the (last) xml editor in focus are computed and printed in the textarea below. (You can open the XPath window either by using the context menu on xml files or directly via the window menu)

It uses the JDK's javax.xml.xpath package which means XPath 1.0 support.

it also supports basic auto completion and localized error messages (in other words: XPathExpressionException.getCause().getLocalizedMessage()).

Github project can be found here. Feel free to use it ;)

Since I haven't bloged for a long time about the OpenGL Pack, here are some notes for the last two update releases.

Beside updates and fixes the only new feature I added is the OpenGL Quicksearch. It uses the NetBeans 6.5 Quicksearch functionality (similar to ctrl + 3 in eclipse) and searches through several OpenGL resources.

currently are five resources available for search:

• OpenGL SDK
  
• OpenGL SDK Extension Registry
• Nvidia Extensions
• ATI/AMD Extensions
• JOGL sources (only when JOGL module installed and used in at least one opened project)

power users can restrict the search to a category with the following prefixes ([prefix  space] searchstring  (space searchstring)*):
gl for GL SDK, ext for GL extensions, nv for Nvidia extensions, ati for ATI/AMD Extensions.

Pictures featuring: Form Designer sample project, GL Quicksearch, updated GLSL 1.3 editor

Changes in 0.5.3:

• Java Applet and Webstart support
• OpenGL Quicksearch
• GLSL editor updated to support GLSL 1.3 (OpenGL 3.0 spec)
• two new NetBeans Form Designer sample JOGL projects
• NetBeans 6.5 and JDK6 are the new minimum requirements
  

Changes in 0.5.2 since last release:

• JOGL distribution has been updated to version 1.1.1
• GLSL linker now does not stop linking shaders on compiler warnings
• the pack should now correctly deploy JOGL and GlueGen's native libraries on all supported platforms (64bit bug)

To download the pack, please visit the plugin portal.

As always, please use the user mailing list for feedback and the issue tracker for bug reports on the project page.

- - - - -

BTW the pack recenty reached the 36k downloads milestone which makes it to the #2 in the most popular plugins category of the plugin portal ;)

The NetBeans OpenGL Pack 0.5.1 has been released. This is a small bugfix release with no major features added.

Changes since last release:

• updated JOGL distribution to v1.1.1rc8
• updated JOGL demos (added gears Applet demo)
• added JOGL API support module for doc and source integration
  
• webstart support for JOGL project templates (just enable it in the project properties)
• improved JOGL deployment code (fixed some issues on 64bit systems)
• bug fixes

To download the pack please visit the plugin portal page.

The OpenGL pack will be soon available via plugin portal update center (Tools | Plugins). This will make our old  "NetBeans OpenGL Pack update center" obsolete (and improve the download speed!). However, we will use the old update center for experimental not final releases in future.

As always, please use the user mailing list for feedback and the issue tracker for bug reports on the project page.

-------

Write once run everywhere [1][2][3][4][5][6] - seems like the platform agnostic JOGL deployment is working ;)

If you develop NetBeans modules you probably also wondered how to add a pre-packaged .nbm file as dependency to your module or how to use modules in multiple suites.

After some search I found that there is currently no official supported way for doing this. The reason is the bidirectional dependency between the suite and each module in the suite. This makes it hard to reuse modules across suites but it is still possible. In this entry I will describe different techniques/hacks to workaround this issue. (Note: technique, hack and workaround in the same sentence ;) )

In general, you can add a dependency to a module when the module you like to depend on is in:

a) your suite

b) the target NetBeans platform (default target platform is the installed IDE)

As mentioned above a) works only for one suite -> only b) is left

option 1. Assemble your own target platform:

point and click

• Either make a copy of NetBeans, or if there's a lot of stuff you don't want, create an empty suite. Then build it and create a zip distribution and unpack that
• Launch the result
• In Tools | Plugins, install the module(s) you want globally (check the "Force install into shared directories" checkbox)
• Shut it down
• In your real IDE, use Tools | NetBeans Platforms to point at the copy of the platform that now has the module you want in it
• Set your suite to build against that. It should pick up the module you installed and all classes in it's public packages

or put it in a build script

option 2. Install the module into your IDE:

point and click

• enable "Force install into shared directories" in Tools | Plugins | Settings
• install your module(s)
• restart NetBeans
  
• your module is now in the extra cluster and part of your IDE
  

or declarative

edit Info/index.xml inside your .nbm file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC "-//NetBeans//DTD Autoupdate Module Info 2.5//EN" "http://www.netbeans.org/dtds/autoupdate-info-2_5.dtd">
<module codenamebase="org.yourorghere.module4" distribution="" downloadsize="0" global="true" homepage="" license="AD9FBBC9" moduleauthor="Michael Bien" needsrestart="false" releasedate="2008/04/07" targetcluster="milkyway">
    <manifest AutoUpdate-Show-In-Client="true" OpenIDE-Module="org.yourorghere.module4" OpenIDE-Module-Implementation-Version="080407" OpenIDE-Module-Java-Dependencies="Java &gt; 1.5" OpenIDE-Module-Name="module4" OpenIDE-Module-Requires="org.openide.modules.ModuleFormat1" OpenIDE-Module-Specification-Version="1.1"/>
    <license name="AD9FBBC9">[NO LICENSE SPECIFIED]
</license>
</module>

alternatively you can add the flags to module/nbproject/project.properties and rebuild/create NBM (if source available)

nbm.is.global=true
nbm.target.cluster=milkyway

Now when you install the module with Tools | Plugins, NetBeans will place the module directly into the installation folder in the new created "milkyway" cluster. With this trick you should be able to use the same library wrapper module in as many suites you want.

Thanks to all on the mailing lists for the help on that topic.

------

I am already looking forward to JSR 277 which will hopefully replace the current implementation of the NetBeans module system. (no there was nothing anounced but this would make defenetive sense)

Note: This entry has been posted on 1. April 2008 and nothing of that below is true :-)

You probably already know a lot of changes are planed for the NetBeans 7.0 release.

One of the bigger changes is tighter integration to the Windows Presentation Foundation for the SWT/JFace rewrite of NetBeans 7.0 similar tho the Eclipse roadmap. The minimum system requirement will rise to windows vista ultimate with a DirectX 10 capable graphics card and a USB stick plugged into your system (swap file for java quickstarter) to render NetBeans 7 in full HD. The primary reason for that was the out of the box Java 6 incompatibility to apple systems (who knows maybe it is compatible with MacOS X but no one will tell you because if he tried installing SE 6 on macs he/she also signed an NDA...) and the issue that many architects simple do not understand the internals of linux distributions (e.g Ubuntu) to install NetBeans.

On older cards or other operating systems JEdit will be started in compatibility mode (motif look&feel and full shell support). The reason for that are the new consumer guidelines and download size limitations of the java 6 Update 10 release. (there is a because of backwards compatibility problems [to build 11] not fixable ArrayIndexOutOfBounds bug in the pack200 implementation - primary reason why the swing renderer does not fit into the NB 7 distribution anymore if started with Java 6 update 10).

This brings several advantages. E.g instead of playing Jake in the browser (update 10 required) NetBeans 7.0 will be capable to render Halo 3 in the editor pane (with full profiler integration and 16x FSAA text overlay). Regular patches will be available via update center.
 
Additionally to that better joystick support is planed. This should improve the navigation through larger projects and replace the "go to declaration" action. You may also activate the force feedback option in the new "user experience" tab in the options dialog to detect the files which causes unit tests to fail. (Note: not available in JEdit compatibility mode but there will be a blinking icon instead)

The higher costs to develop NetBeans 7.0 make a complete free distribution not feasible but it will be still free for opensource developers (but not without limitations e.g UML diagrams will be limited to only two kind of widgets "hack" and "ship" while maintaining 100% SSP compatibility).

The NetBeans OpenGL Pack is slowly evolving to a set of useful integrated tools, demo applications and editors for Java OpenGL development.

The features currently included are:

• Editor for the OpenGL Shading Language (GLSL) with compiler error annotation, code folding, syntax highlighting, auto completion and documentation.
  
• Easy access to the GLSL compiler and linker of your graphics driver integrated in the editor
  
• OpenGL Capabilities Viewer
• Integration of JOGL GUI components into the Matisse GUI builder
• JOGL project templates
• Ready to run JOGL demos and examples of the OpenGL Programming Guide (also known as Red Book) 

We are currently fixing the last bugs and hope to build the first release soon. The early access version is available via Update Center in the download section of the project page.

------------
The project has also been recently mentioned in the NetBeans Magazine [issue four] covering NetBeans 6 features. Geertjan Wielenga interviewed me about the GLSL editor in the Schliemann section of the magazine.