I’m excited to announce that my latest RefCard for DZone has been released. In a Q&A session with Lyndsey Clevesy, I discuss the card’s focus on helping developers get started with the Google App Engine for Java.
The RefCard is intended to be a jumping off point and provides information about and links to the frameworks that now support GAE as well as external resources to help you dive deep if this new arena of cloud computing. Grab a copy and send me feedback.
At several stops on the NFJS tour, I’ve been asked about some of the minor issues in running JPS and VisualVM on the Windows platform. The primary problem is that the processes in JPS and VisualVM are listed as:
<Unknown Application> (pid ###)
I have been successful in working around this by renaming the hsperfdata temp directory as noted in a separate thread on the Sun forums.
The BTrace plugin for VisualVM, a tool that I love to showcase at No Fluff Just Stuff and user group events, has been updated to version 0.2.2. This solves an issue that prevented it from working on a Mac. Be sure to uninstall any prior version from your VisualVM prior to installing 0.2.2 to get the full benefit of the fix.
I’ve been battling getting jmap working properly on a new Mac. I followed the invaluable instructions my NFJS colleague, Ken Sipe, provided for the Rhino engine inclusion. However, I was still encountering an odd issue where I couldn’t get jmap to attach to a Java process.
First, I would start a simple helloworld Java application.
java com.ambientideas.HelloWorldJava Hello Java World!
Then I would get the PID:
jps 11554 HelloWorldJava 11573 Jps
Then I tried to jmap it, but got the following error
jmap -heap 11554 Attaching to process ID 11791, please wait... attach: task_for_pid(11791) failed (5) Error attaching to process: Error attaching to process, or no such process
So I thought, “These memory tools often require root access, so I’ll sudo it.” When I did, I got the following equally cryptic error:
sudo jmap -heap 11791 Attaching to process ID 11791, please wait... Exception in thread main java.lang.RuntimeException: gHotSpotVMTypes was not initialized properly in the remote process; can not continue at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:111) at sun.jvm.hotspot.HotSpotTypeDataBase.&amp;amp;amp;amp;amp;lt;init&amp;amp;amp;amp;amp;gt;(HotSpotTypeDataBase.java:68) at sun.jvm.hotspot.MacOSXTypeDataBase.&amp;amp;amp;amp;amp;lt;init&amp;amp;amp;amp;amp;gt;(MacOSXTypeDataBase.java:35) at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:560) at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:481) at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:319) at sun.jvm.hotspot.tools.Tool.start(Tool.java:146) at sun.jvm.hotspot.tools.JMap.main(JMap.java:128)
I though, “Hmmmm, this looks a lot like a java version mismatch. Let’s check which java version I’m running.” I had javac compiled the HelloWorld against Java 6, so was expecting a matching java. I made a quick stop into the Java Preferences pane. Yup. Java 6 set as the first-priority choice.
Let’s check the command line too…
java -version java version 1.6.0_13 Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211) Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)
Excellent. Just what I expected. Java 6. But a little doubt crept into my mind. What if the sudo’ed invocation were finding a different Java. That didn’t sound plausible, but yet was still worth checking out.
sudo java -version java version 1.5.0_19 Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_19-b02-304) Java HotSpot(TM) Client VM (build 1.5.0_19-137, mixed mode, sharing)
Aha! There’s the culprit. So somehow, sudo’ed commands are running Java 5, even though I’ve set a preference for 6 in my profile. I searched the web high and low for a way to specify the Java version for “sudo” specifically, but could find no trace. I even tried the -E flag to maintain the caller’s environment variables, as is possible on Linux distributions. Not implemented on Mac.
I did a quick check of my path and a “which java”. Both ultimately routed to Java 6. I also had JAVA_HOME and JAVA_VERSION set just fine.
set | grep JAVA JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home JAVA_VERSION=1.6 echo $PATH /Applications/Dev/btrace/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin: /usr/X11/bin:./: /Users/mccm06/scripts:/Applications/Dev/apache-ant/bin: /Applications/Dev/apache-maven/bin
So I turned to harness the wisdom of one of the really smart guys (that uses a Mac) I’m privileged to hang out with in Denver, Fred Jean. Fred said, “Try adding the 1.6 java explicitly to the front of your path rather than letting Java Preferences pane be the sole controller.” I opened up ~/.bash_profile (alternatively you could edit ~/.bash_rc)and slammed in the following:
export PATH=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Commands:$PATH
I opened a new terminal to pick up the shell changes. Then I started checking if this fixed the issue in reverse order, starting with the sudo:
sudo java -version java version 1.6.0_13 Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211) Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)
Looking good! Now let’s get back to our core goal, jmap.
sudo jmap -heap 5951
Attaching to process ID 5951, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11.3-b02-83
using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 88080384 (84.0MB)
NewSize = 21757952 (20.75MB)
MaxNewSize = 43581440 (41.5625MB)
OldSize = 64159744 (61.1875MB)
NewRatio = 7
SurvivorRatio = 6
PermSize = 21757952 (20.75MB)
MaxPermSize = 88080384 (84.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 19070976 (18.1875MB)
used = 2351616 (2.24267578125MB)
free = 16719360 (15.94482421875MB)
12.330863402061855% used
Eden Space:
capacity = 16384000 (15.625MB)
used = 2351616 (2.24267578125MB)
free = 14032384 (13.38232421875MB)
14.353125% used
From Space:
capacity = 2686976 (2.5625MB)
used = 0 (0.0MB)
free = 2686976 (2.5625MB)
0.0% used
To Space:
capacity = 2686976 (2.5625MB)
used = 0 (0.0MB)
free = 2686976 (2.5625MB)
0.0% used
concurrent mark-sweep generation:
capacity = 64159744 (61.1875MB)
used = 0 (0.0MB)
free = 64159744 (61.1875MB)
0.0% used
Perm Generation:
capacity = 21757952 (20.75MB)
used = 4716880 (4.4983673095703125MB)
free = 17041072 (16.251632690429688MB)
21.678878600338855% used
Thanks to a combination of Ken Sipe, Fred Jean, a few good guesses and some investigative work, the problem is solved! I’m longwindedly calling this the “java version is different for sudo-initiated processes on Mac” so that the next person Googling for this using similar keywords will hopefully find this tutorial.
As a closing note, I often get asked how I set my environment variables for Java so that Spotlight-launched, Quicksilver-launched, Dock-launched, and Terminal-launched processes can all see the variables (yes, there’s some odd pitfalls about that on a Mac due to the process parents being different). The short answer is to create a text file called /private/etc/launchd.conf and put the following (applicable portions, based on your tool usage) contents in it:
setenv JAVA_VERSION 1.6 # A) Some process-launchers will cause their child to default to 1.5 if using /Library/Java/Home # even if Java Preferences pane says 1.6 at the top # setenv JAVA_HOME /Library/Java/Home # B) Alternate: Force Java 1.6 as Java Home setenv JAVA_HOME /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home setenv GROOVY_HOME /Applications/Dev/groovy setenv GRAILS_HOME /Applications/Dev/grails setenv NEXUS_HOME /Applications/Dev/nexus/nexus-webapp setenv JRUBY_HOME /Applications/Dev/jruby # Set up Ant setenv ANT_HOME /Applications/Dev/apache-ant setenv ANT_OPTS -Xmx512M # Set up Maven setenv MAVEN_OPTS -Xmx1024M setenv M2_HOME /Applications/Dev/apache-maven
As we have learned the hard way, the only pitfall is that the java version specified in launchd.conf is not honored by sudo’ed java invocations.
Drop me an email if you’ve seen this issue and this tutorial helps you solve it.
I’m very pleased to announce that my DZone Maven RefCard was released this week ahead of schedule, and JavaLobby did a little introductory interview for the launch. A handful of folks contributed to the early alpha reviews and I want to acknowledge their inputs: Ken Sipe, Tim O’Brien, Chris Maki, Tim Berglund, and Jason van Zyl.
The timing was great, as I did a Mastering Maven talk at the San Diego Java Users Group (SDJUG) on Tuesday night. A handful of the attendees came with printed copies of the RefCard in hand!
I’m always looking for ways to contribute to the Maven community, and this was by far the most fun I’ve had building materials to promote this unique Convention over Configuration build tool.
Maven 3.0, a mostly-backwards compatible, but significantly improved and extensible version of Maven is developing very quickly. Jason van Zyl hosted a Maven Meetup at their offices in Mountain View in March of this year. Lots of deep information about Maven 2.0 and 3.0 was shared, and videos of some of the sessions are now starting to be posted to the Vimeo web site. The Maven 3.0 video is especially insightful:
Jason van Zyl on Maven 3 from Sonatype on Vimeo.
Here are the takeaway points, highly distilled to 140 proof, for those without time to watch the video:
If you want to get access to the Maven 3.0 code, you can view the list of SVN checkout URLs here (or here for the hardcore Git lovers), or just view the source in your browser here. If you want early access to the binaries, you can download them from Apache.
If you are running a Continuous Integration server such as Hudson, you’ll want to consider routing your SureFire outputs to the console so that they’ll appear in the build-report logs. If you leave SureFire at its default, it will output each test’s success or failure to an individual test XML and TXT file, but those are likely not in an exposed directory on your CI server. If instead, you route the output to the console, it will get reported in your failure emails that your CI server is capable of sending.
Just pass the useFile=false parameter on the command line or set it in the plugin config section of your pom.xml.
mvn test -Dsurefire.useFile=false
Before:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.ambientideas.AppTest
Hello World! This is a JUnit test!
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec <<< FAILURE!Results :Failed tests:testApp(com.ambientideas.AppTest)Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
After:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.ambientideas.AppTest
Hello World! This is a JUnit test!
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec <<< FAILURE!
testApp(com.ambientideas.AppTest) Time elapsed: 0.014 sec <<< FAILURE!
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at com.ambientideas.AppTest.testApp(AppTest.java:37)
Results :Failed tests:testApp(com.ambientideas.AppTest)Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
Tonight was the Denver Open Source User’s Group March meeting.
First up was Clark Hobbie on the CLIP IPC Library.
First, Clark addressed the question,”Why do we need an IPC library?” . He purports you need IPC “anytime you access something outside your JVM and need to share it in a controlled and coordinated manner with another client”. Clark says that CLIP was created as an answer to the cryptic and verbose shared memory classes in the JDK.
A brief outline of what he covered in his slides is as follows:
Clark did a great job on his slides with funny anecdotes, images for analogies, and clear verbal examples of IPC types (props to World of Warcraft).
Talend ETL Tool
Second up was Tim Berglund speaking on the open source Talend Open Studio ETL (Extract Transform and Load) system. This French startup company is attempting to create a new price point for ETL tools with the now-common OSS business model, selling support and training while giving away the core product for free.
He neatly said this is a talk for non-DBAs but rather developers that need to work with databases. Tim admitted that there are a few negatives to the otherwise great Talend tool. Those are: the JAR is 70MB, the error messages have a French accent, and Mac Eclipse support is a work in progress.
The visual designer has a lot of off-the-shelf transforms. We also saw XML, Excel files, 10+ DB brands, and CSVs as just some of the data sources.
It was a fun set of slides that were in the vein of Slideology.
This post succinctly sums up significant research that I did over the last few days on the available means of setting environment variables for a Windows Domain Group of users at logon time. A Java Swing-based desktop application I’m helping a client with reads environment variables for its connectivity settings (server IP, etc.), so, it is logical that there needs to be a way to distribute these environment variables all all users from the Domain Controller. Below are the results of that research.
My research says there are four ways to do this. I started at the Microsoft Logon Script documentation pages and fanned out from there.
Login batch file (.BAT) scripts are just a temporary instance of a CMD window, and the environment variables set in there go away as soon as the login window closes.
set MYVAR=MyValue
Won’t work for the aforementioned reason.
So, alternatively, I can try to set the variable via directly writing to the registry like so for a System Environment Variable:
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment" /v MYVAR /t REG_EXPAND_SZ /d MyValue
or to the User Environment Variables like so:
reg add HKCU\Environment /v MYVAR /t REG_EXPAND_SZ /d MyValue
The drawback here is that the variables, though written to registry, are not read until the next login for all I can see. A new CMD window shows no trace of them until the user re-logs-in.
With a Visual Basic Script (VBS) login script, you can use a more programmatic method to access the environment variables. This is looking like my most viable approach. This example would append to the end of PATH.
Set WSHShell = WScript.CreateObject("WScript.Shell")Set WshEnv = WshShell.Environment("SYSTEM")WshEnv("Path") = WshEnv("Path") & ";M:\DB\whatever\"
This example would just set the variable.
Set WSHShell = WScript.CreateObject("WScript.Shell")Set WshEnv = WshShell.Environment("SYSTEM")WshEnv("MYVAR") = "MyNewValue"
This approach yields variables that are immediately available via a CMD window. No reboot is required like the batch file registry writes.
ADM files are a way to expose custom functionality of settings to the Group Policy Editor. It seems tricky to get them installed and visible on the domain controller so I’m jumping over this option.
Microsoft Support TechNet Reference on ADM File Locations.
Another article about ADM files and using them to set Registry settings.
Tom’s Hardware on ADM Files.
---- set.adm ---- CLASS MACHINE CATEGORY "Environment" POLICY "Self dfined variables" KEYNAME "SYSTEM\CurrentControlSet\Control\Session Manager\Environment" PART "Set MyVar1 =" EDITTEXT DEFAULT "MyValue1" VALUENAME MyVar1 ; EXPANDABLETEXT ; add expandabletext if it can contain Variables itself END PART END POLICY END CATEGORY ---- set.adm ----
Windows Server 2008 has a new feature called the Environment Extensions for the Group Policy Preferences. It allows you to conveniently set what otherwise required complex batch scripts. The new items exposed include registry values, environment variables, and more. A quick how-to guide is available here.
I can’t use this option because my clients don’t have Windows Server 2008.
After many sessions of debugging and even memory inspection not yielding fruit, I’ve finally discovered what caused my previous RESTful demo apps to have sporadic behavior, in addition to the memory leak of the synchronous call.
Today’s problem solving summary:
ld: -filelist file not found: <Project Path>
This bit me because my “Add Contestant” function was adding the contestant and returning void. I now return a string of “Success” to satisfy this requirement. The observable failure is very quiet, which made it so hard to debug; every other web service call appears valid, but never actually makes a call across the wire. This was proven via TCPDump. All the web service call callbacks are properly called in the Objective C side of the program. It just silently failed and passed back a null payload in the response data structure.
All the code for this project is available on GitHub and will be updated with the results of the adjustments based on the above findings by tomorrow.