In my last post, I talked about a shortcoming of Log4j with a thread leak. I took a look at the Log4j 2 source code to see if it suffers from a similar issue.
The Log4j 2 implementation that checks for changes to the configuration file is handled by FileConfigurationMonitor.java. No threads are created to monitor changes, and from my testing, I saw no thread leaks re-deploying my application to Tomcat.
The new behavior calls FileConfigurationMonitor.java checkConfiguration method during each logging call. If the time since the last check has exceeded your monitorInterval passed into the logging configuration and the file has been modified on the disk, the configuration is reloaded.
The behavior works as expected, but I encountered and initial issue by setting my refresh interval to 1 second...the minimum amount of time one can check for changes to the configuration file is hard-coded at 5 seconds. "But Log4J, what if I have a requirement where I need to apply the change within 1 second????" Five seconds seems like enough time between checks, and I could probably write my own implementation to get rid of the hard-coded limitation and plug it in somehow. But, why choose 5 seconds as the minimum and give no way to change it?
From my brief experience with Log4j 2, I like the APIs, ease of configuration settings, and will look forward to the promise of performance increases (those are always welcome no matter how small in any software). And best of all, no thread leak!
Friday, September 27, 2013
Log4j And Web Application Thread Leaks
Let's breathe some life back into this blog!
I'll start off by talking about an issue and solution I recently came across.
The Apache Log4J framework provides a way to re-configure logging on-the-fly by editing the configuration file. The implementation spawns a thread (FileWatchdog.java) that monitors the configuration file for changes. FileWatchdog never checks Thread.isInterrupted() . Instead, it catches the interruption making the FileWatchdog thread live until the jvm is stopped.
The problem with this implementation (in version 1.2.16 I tested) is there is no easy way to stop the thread--some people have written that stopping it is impossible. In a web application environment where you may re-deploy offten, the end result is several threads that never stop along with memory leaks on any held resources in the running threads.
Tomcat 7 provides us a warning:
With a simple test case, I discovered an easy way to forcibly kill these threads:
The downside with this approach is you have call the deprecated Thread.stop() method. The upside is this method is still available in Java 7. Placing the call in a ServletContextListener allows the container to stop these threads during the undeploy process.
Tomcat now shows no warning the tread was left running:
Using jvisualvm or any other tool confirms the thread was stopped.
It seems like such a quick fix (e.g. || isInterrupted()) FileWatchdog could throw in there--I'm not sure why this hasn't been addressed. Given the design decisions of Log4J to really have one instance per jvm, the current implementation makes sense. However, it's common for individual web apps to configure their logging independent of any other web apps or global logging. In addition it's just good coding standards for frameworks to provide clean startup/shutdown apis. Maybe this is been addressed in the upcoming Log4j 2 .
I'll start off by talking about an issue and solution I recently came across.
The Apache Log4J framework provides a way to re-configure logging on-the-fly by editing the configuration file. The implementation spawns a thread (FileWatchdog.java) that monitors the configuration file for changes. FileWatchdog never checks Thread.isInterrupted() . Instead, it catches the interruption making the FileWatchdog thread live until the jvm is stopped.
The problem with this implementation (in version 1.2.16 I tested) is there is no easy way to stop the thread--some people have written that stopping it is impossible. In a web application environment where you may re-deploy offten, the end result is several threads that never stop along with memory leaks on any held resources in the running threads.
Tomcat 7 provides us a warning:
Sep 27, 2013 10:29:36 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/log4jtest] appears to have started a thread named [FileWatchdog] but has failed to stop it. This is very likely to create a memory leak.
With a simple test case, I discovered an easy way to forcibly kill these threads:
/**
* @see ServletContextListener#contextDestroyed(ServletContextEvent)
*/
@SuppressWarnings("deprecation")
public void contextDestroyed(ServletContextEvent sce) {
for (Thread t : Thread.getAllStackTraces().keySet()) {
if (t.getName().equals("FileWatchdog")) {
System.out.println("Found FileWatchdog thread. Killing it");
t.stop();
}
}
System.out.println("contextDestroyed");
}
The downside with this approach is you have call the deprecated Thread.stop() method. The upside is this method is still available in Java 7. Placing the call in a ServletContextListener allows the container to stop these threads during the undeploy process.
Tomcat now shows no warning the tread was left running:
INFO: Reloading Context with name [/log4jtest] has started Found FileWatchdog thread. Killing it contextDestroyed Sep 27, 2013 10:40:18 AM org.apache.catalina.core.StandardContext reload
Using jvisualvm or any other tool confirms the thread was stopped.
It seems like such a quick fix (e.g. || isInterrupted()) FileWatchdog could throw in there--I'm not sure why this hasn't been addressed. Given the design decisions of Log4J to really have one instance per jvm, the current implementation makes sense. However, it's common for individual web apps to configure their logging independent of any other web apps or global logging. In addition it's just good coding standards for frameworks to provide clean startup/shutdown apis. Maybe this is been addressed in the upcoming Log4j 2 .
Friday, April 16, 2010
SWT Table Images And Transparency
I recently came across a Windows platform issue using images inside an SWT table. A little research turned up this long-standing open SWT bug.
The problem results when you change the background color of a row in a table. I like to alternate colors on the rows in my tables, so I create a LabelProvide like this:
Of course, you would actually want to return some real values in those methods.
Using a JFace TableViewer allows the table to be decorated with the listener via the setLabelProvider method.
Under Linux, SWT uses GTK native widgets for table. An example of one of my tables looks like this:
Make sure your monitor is set up so you see each row is a different shade--the first row is a deep gray, and the second row is black.
The table looks fine and all of the images ( transparent png files ) are correctly displayed. Now look at the same application on Windows 7.
Notice the images have black backgrounds now which is the background of the table. The transparency mask in the png is ignored--annoying and ugly. Look closely at the circled area to see the problem.
Luckily, with a little help from old posts in that bug and some trial and error, I came up with an easy fix that works for Windows and Linux. Using the same label provider as above, we can get the row color during the SWT.EraseItem event and erase the area with the same color used for the row:
And the result on Windows 7:
The result is how I want it. The images blend in correctly. I'm not sure what the real problem is here--is it Windows or Eclipse? Either way, the fix works for me, and I can continue on with development.
The problem results when you change the background color of a row in a table. I like to alternate colors on the rows in my tables, so I create a LabelProvide like this:
private class PortfolioViewLabelProvider extends LabelProvider implements ITableLabelProvider, ITableColorProvider {
@Override
public Color getBackground(Object element, int columnIndex) {
return null;
}
public Image getColumnImage(Object obj, int index) {
Image retImage = null;
return retImage;
}
public String getColumnText(Object obj, int index) {
String retVal = "";
return retVal;
}
@Override
public Color getForeground(Object element, int columnIndex) {
return null;
}
}
Of course, you would actually want to return some real values in those methods.
Using a JFace TableViewer allows the table to be decorated with the listener via the setLabelProvider method.
Under Linux, SWT uses GTK native widgets for table. An example of one of my tables looks like this:
Make sure your monitor is set up so you see each row is a different shade--the first row is a deep gray, and the second row is black.
The table looks fine and all of the images ( transparent png files ) are correctly displayed. Now look at the same application on Windows 7.
Notice the images have black backgrounds now which is the background of the table. The transparency mask in the png is ignored--annoying and ugly. Look closely at the circled area to see the problem.
Luckily, with a little help from old posts in that bug and some trial and error, I came up with an easy fix that works for Windows and Linux. Using the same label provider as above, we can get the row color during the SWT.EraseItem event and erase the area with the same color used for the row:
topPortfoliosViewer.getTable().addListener(SWT.EraseItem, new Listener() {
@Override
public void handleEvent(Event event) {
event.gc.setBackground(plp.getBackground(event.item.getData(), event.index));
event.gc.fillRectangle(event.getBounds());
}
});
And the result on Windows 7:
The result is how I want it. The images blend in correctly. I'm not sure what the real problem is here--is it Windows or Eclipse? Either way, the fix works for me, and I can continue on with development.
The Future Of OpenSolaris...Still Unwritten and Uncertain
I've been following the activities of the OpenSolaris project for some time with interest. With the acquisition of Sun by Oracle, it's future is still uncertain. Since my last entry on the subject, I have only played with it a few times inside a virtual machine.
I remember back when I was in college and OS/2 Warp came out. It was so cool and I was immediately converted. Despite the coolness factor, the conversion was short lived. I don't remember my reasoning for abandoning OS/2--maybe it was my introduction to Linux in a computer lab or the lack of OS/2's ability to satisfy my computer needs compared to Windows or Linux. Whatever the reason, I can't help but recall that experience as I now look at and ponder the fate of OpenSolaris.
I don't think OpenSolaris has a large enough community outside of full-time, paid developers to support the operating system's advancement. Sure, there are a few different distributions out there, but these deal primarily with packaging the kernel--not low-level kernel development. As with OS/2 warp, a few cool technologies found in no other operating system (i.e. DTrace and ZFS ) will not be enough to grow the community and keep the system alive. Without a large and fully open community, a code base the size of an operating system needs some major company-backed financing to grow.
Only time will tell the true fate of OpenSolaris. I think some years down the road I'll look back on my OpenSolaris experience and remember the brief experience with a smile and wonder what may have been.
I remember back when I was in college and OS/2 Warp came out. It was so cool and I was immediately converted. Despite the coolness factor, the conversion was short lived. I don't remember my reasoning for abandoning OS/2--maybe it was my introduction to Linux in a computer lab or the lack of OS/2's ability to satisfy my computer needs compared to Windows or Linux. Whatever the reason, I can't help but recall that experience as I now look at and ponder the fate of OpenSolaris.
I don't think OpenSolaris has a large enough community outside of full-time, paid developers to support the operating system's advancement. Sure, there are a few different distributions out there, but these deal primarily with packaging the kernel--not low-level kernel development. As with OS/2 warp, a few cool technologies found in no other operating system (i.e. DTrace and ZFS ) will not be enough to grow the community and keep the system alive. Without a large and fully open community, a code base the size of an operating system needs some major company-backed financing to grow.
Only time will tell the true fate of OpenSolaris. I think some years down the road I'll look back on my OpenSolaris experience and remember the brief experience with a smile and wonder what may have been.
Saturday, November 28, 2009
GNOME Composting in OpenSolaris
If your graphics card supports it, OpenSolaris will turn on the 3D effects when you login. It does this by enabling the compiz window manager which replaces the default GNOME window manager, metacity.
I only recently discovered that metacity also supports composting. It doesn't offer the feature list of compiz, but, it does provide nice drop shadows and task switcher.
To use composting in metacity, set Visual Effect to None in the System/Preferences/Appearance dialog box. Apply the changes and your desktop will be running metacity.
Next, run this command to enable composting in metacity:
and you can run this command to turn it off:
Here's a simple snapshot showing the shadows:
If you really like tweaking your desktop, compiz works very well and provides lots of features. If you're happy with drop shadows and a nice tab switcher, metacity+compositing will do the job nicely.
I only recently discovered that metacity also supports composting. It doesn't offer the feature list of compiz, but, it does provide nice drop shadows and task switcher.
To use composting in metacity, set Visual Effect to None in the System/Preferences/Appearance dialog box. Apply the changes and your desktop will be running metacity.
Next, run this command to enable composting in metacity:
gconftool-2 -s /apps/metacity/general/compositing_manager -t bool true
and you can run this command to turn it off:
gconftool-2 -s /apps/metacity/general/compositing_manager -t bool false
Here's a simple snapshot showing the shadows:
If you really like tweaking your desktop, compiz works very well and provides lots of features. If you're happy with drop shadows and a nice tab switcher, metacity+compositing will do the job nicely.
Sunday, October 11, 2009
OpenSolaris Performance Issues Resolved
I've been playing with some soon-to-be-released operating systems--notably Open Suse 11.2 and OpenSolaris 2010.02. I always really enjoy OpenSolaris when I run it in a virtual machine under Linux and I enjoy working on Solaris workstations at my employment. I've tried OpenSolaris on and off over the past several months on my home computers. When I've installed it directly onto a partition and run it directly on bare metal, I've noticed performance issues and I've gone back to Linux as the primary OS.
In my latest endeavors of running OpenSolaris, I noticed that the disk io seemed to be the performance killing culprit. Doing simple disk io tests under both Linux and OpenSolaris, Linux was just destroying the io numbers I was getting under OpenSolaris. My test was a simple "dd if=/dev/zero of=test bs=4048 count=100000 conv=fsync"--nothing fancy, but enough to demonstrate an issue. Linux numbers were around 60 MB/s while OpenSolaris numbers were in the 15-20 MB/s. No settings helped out the OpenSolaris numbers--dd block sizes, file system block sizes, zfs settings, etc. Interestingly enough, on an older computer, the disk io was just fine under OpenSolaris, so why was it so slow on newer/faster computer?
I was almost ready to give up and wipe the OpenSolaris partition once more, but I had one last area I had not tried. I remembered the SATA settings that are within the bios settings of my Intel motherboard. I changed the SATA mode from Legacy to AHCI. Sure enough, my OpenSolaris hard drive speeds were now on par with Linux. Inspection of the loaded modules showed that OpenSolaris had loaded the ahci driver. Additional online research shows that one should set the mode to AHCI for OpenSolaris.
I'm definitely happy with the results on all 4 home machines. 3 diverse desktop models and 1 laptop are running very smoothly. I'll plan on detailing some lessons learned and tutorials in the near future.
In my latest endeavors of running OpenSolaris, I noticed that the disk io seemed to be the performance killing culprit. Doing simple disk io tests under both Linux and OpenSolaris, Linux was just destroying the io numbers I was getting under OpenSolaris. My test was a simple "dd if=/dev/zero of=test bs=4048 count=100000 conv=fsync"--nothing fancy, but enough to demonstrate an issue. Linux numbers were around 60 MB/s while OpenSolaris numbers were in the 15-20 MB/s. No settings helped out the OpenSolaris numbers--dd block sizes, file system block sizes, zfs settings, etc. Interestingly enough, on an older computer, the disk io was just fine under OpenSolaris, so why was it so slow on newer/faster computer?
I was almost ready to give up and wipe the OpenSolaris partition once more, but I had one last area I had not tried. I remembered the SATA settings that are within the bios settings of my Intel motherboard. I changed the SATA mode from Legacy to AHCI. Sure enough, my OpenSolaris hard drive speeds were now on par with Linux. Inspection of the loaded modules showed that OpenSolaris had loaded the ahci driver. Additional online research shows that one should set the mode to AHCI for OpenSolaris.
I'm definitely happy with the results on all 4 home machines. 3 diverse desktop models and 1 laptop are running very smoothly. I'll plan on detailing some lessons learned and tutorials in the near future.
Wednesday, July 1, 2009
OpenSolaris 2009.06 Progressing Quickly
Previously, I spoke of my OpenSolaris experiences. Over the past week, I've been on vacation, and I decided to try the latest release out to see how it's moving along. I installed version 2009.06 over my laptops Fedora installation. Installation was once again very simple--just a few questions, and it was off installing. I typically choose to use the entire drive as a single ZFS pool. OpenSolaris does not support disk encryption out of the box like Fedora or other Linux distributions, but work is underway in this area.
After the initial install, I immediately upgraded to the development repositories to bring in the more more bleeding-edge stuff( Fedora is pretty bleeding edge too ). An update and a reboot into the new boot environment, and I was up and running.
I discovered there are a few really nice new features and surprises in the latest builds.
First off is the integration of the new sound system--Boomer for OpenSolaris. This is now all integrated with the latest development builds. Sound playback worked wonderfully on my Laptop. In flash videos and playback in general under the previous sound system, sliding the volume control up and down caused serious pauses and stutter. The new sound system works much more smoothly and I noticed no more pauses or stutters while changing the volume. I have noticed that I get no headphone sound right now on my laptop, but overall, the new sound system is a nice improvement.
The next thing I noticed was the brightness control finally works on my laptop. The hotkeys don't work, but the Gnome brightness applet allows me to control the brightness. This was a major drawback of previous builds, since the only solution seemed to be a Linux live cd boot to fix the brightness.
Just a few days ago, I notice that my other major gripe has been fixed too--drivers for my two home computers wireless cards are now available! I'll be testing this out soon, since this means I have the option to put OpenSolaris on two home computers without hardware upgrades.
As I've moved around to different Wifi points, the Network Manager has had no problems connecting up to the different access points automatically using WPA2 and WEP encryption. I've had fewer issues with the automated OpenSolaris Network manager than the Fedora network manager.
As far as usability, my laptop experience has been far more pleasant than previous versions of OpenSolaris. With Eclipse Galileo builds supporting OpenSolaris ( 32 bit ) as a first class citizen, my development IDE needs are met.
Off the top of my head, there are three areas still lacking when comparing OpenSolaris to modern Linux distributions.
The first area is one I mentioned earlier--ZFS/filesystem encryption. I prefer, and many companies now require laptops to have encrypted hard drives. At the rate things are moving in the OpenSolaris development, I suspect this will no longer be an issue in the months to come.
The second area lacking is multimedia support. The issue here is a licensing issue where OpenSolaris can not ship with codecs to provide mp3 playback or encrypted dvd playback support. Attempting to play such media pops up an information box allowing one to get the mp3 from Fluendo for free or to pay for the suite of codecs ( I'm not even sure if this package will allow dvd playback). An alternative solution is to download the latest MPlayer source code and compile the code. There were a few issues getting MPlayer to compile, but drop me a line if you need help. Once MPlayer was compiled with Xv support, I was able to play back some dvd's though I did see an annoying video stutter happen every few seconds. I suspect one should also be able to compile the gstreamer plugins that provide the proper mp3 and video codecs too, though I've not gotten to try that yet.
The final area is performance. Linux has made great strides in the last year in performance. Boot time races between Ubuntu and Fedora have made a very noticeable difference for Linux users who run laptops or desktops and pay attention to the boot up time. OpenSolaris is not quite there on startup time. It still takes a bit longer to get to a login screen compared to Fedora 11 or Ubuntu 9.04, however, it's not terrible. The first OpenSolaris bootup after install is a lot longer than subsequent boot times, so don't give up right away if startup times are important to you. Shutdowns are fast enough to not be an issue for me. The desktop is very responsive, though I do see some waits when say trying to open a new terminal while un-tarring a large file. Linux has been tuned to provide a snappy desktop response in most distributions now, I suspect there are some kernel knobs in the OpenSolaris kernel or ZFS settings to help out here too. For the most part though, I find OpenSolaris as responsive now as I find Linux, but I think Linux still has a slight out-of-the-box advantage here.
OpenSolaris 2009.06+ is already much better that the 2008.11 version I tried earlier. Six months has brought in new drivers and new functionality for the desktop. As Oracle hopefully continues to support the effort, I think the future is holding a definite spot for OpenSolaris. Give it a shot and try it out.
After the initial install, I immediately upgraded to the development repositories to bring in the more more bleeding-edge stuff( Fedora is pretty bleeding edge too ). An update and a reboot into the new boot environment, and I was up and running.
I discovered there are a few really nice new features and surprises in the latest builds.
First off is the integration of the new sound system--Boomer for OpenSolaris. This is now all integrated with the latest development builds. Sound playback worked wonderfully on my Laptop. In flash videos and playback in general under the previous sound system, sliding the volume control up and down caused serious pauses and stutter. The new sound system works much more smoothly and I noticed no more pauses or stutters while changing the volume. I have noticed that I get no headphone sound right now on my laptop, but overall, the new sound system is a nice improvement.
The next thing I noticed was the brightness control finally works on my laptop. The hotkeys don't work, but the Gnome brightness applet allows me to control the brightness. This was a major drawback of previous builds, since the only solution seemed to be a Linux live cd boot to fix the brightness.
Just a few days ago, I notice that my other major gripe has been fixed too--drivers for my two home computers wireless cards are now available! I'll be testing this out soon, since this means I have the option to put OpenSolaris on two home computers without hardware upgrades.
As I've moved around to different Wifi points, the Network Manager has had no problems connecting up to the different access points automatically using WPA2 and WEP encryption. I've had fewer issues with the automated OpenSolaris Network manager than the Fedora network manager.
As far as usability, my laptop experience has been far more pleasant than previous versions of OpenSolaris. With Eclipse Galileo builds supporting OpenSolaris ( 32 bit ) as a first class citizen, my development IDE needs are met.
Off the top of my head, there are three areas still lacking when comparing OpenSolaris to modern Linux distributions.
The first area is one I mentioned earlier--ZFS/filesystem encryption. I prefer, and many companies now require laptops to have encrypted hard drives. At the rate things are moving in the OpenSolaris development, I suspect this will no longer be an issue in the months to come.
The second area lacking is multimedia support. The issue here is a licensing issue where OpenSolaris can not ship with codecs to provide mp3 playback or encrypted dvd playback support. Attempting to play such media pops up an information box allowing one to get the mp3 from Fluendo for free or to pay for the suite of codecs ( I'm not even sure if this package will allow dvd playback). An alternative solution is to download the latest MPlayer source code and compile the code. There were a few issues getting MPlayer to compile, but drop me a line if you need help. Once MPlayer was compiled with Xv support, I was able to play back some dvd's though I did see an annoying video stutter happen every few seconds. I suspect one should also be able to compile the gstreamer plugins that provide the proper mp3 and video codecs too, though I've not gotten to try that yet.
The final area is performance. Linux has made great strides in the last year in performance. Boot time races between Ubuntu and Fedora have made a very noticeable difference for Linux users who run laptops or desktops and pay attention to the boot up time. OpenSolaris is not quite there on startup time. It still takes a bit longer to get to a login screen compared to Fedora 11 or Ubuntu 9.04, however, it's not terrible. The first OpenSolaris bootup after install is a lot longer than subsequent boot times, so don't give up right away if startup times are important to you. Shutdowns are fast enough to not be an issue for me. The desktop is very responsive, though I do see some waits when say trying to open a new terminal while un-tarring a large file. Linux has been tuned to provide a snappy desktop response in most distributions now, I suspect there are some kernel knobs in the OpenSolaris kernel or ZFS settings to help out here too. For the most part though, I find OpenSolaris as responsive now as I find Linux, but I think Linux still has a slight out-of-the-box advantage here.
OpenSolaris 2009.06+ is already much better that the 2008.11 version I tried earlier. Six months has brought in new drivers and new functionality for the desktop. As Oracle hopefully continues to support the effort, I think the future is holding a definite spot for OpenSolaris. Give it a shot and try it out.
Subscribe to:
Posts (Atom)