The Performance Test

Now to the real business, how did I use it?, I have a Web application that runs in Tomcat 6 and connects to MySQL database which I needed to performance test. I utilized the following tools:

1. VisualVM
2. Apache JMeter
3. JMeter Plugins
4. Tomcat 6 running locally with APR
5. MySQL 5 running locally

The test machine specs are: Intel Core 2 Duo T9300 @ 2.5GHz with 4.00GB RAM.

The Results

Initial tests using 50 JMeter user have highlighted a code problem that was blocking a number of Threads. I took a thread dumb using VisualVM during the peak load. The thread dumps indicated an inefficient implementation of XML de-serialization which I then revisited and fixed, below is a comparison of response times vs threads before and after the fix:

Before

After

As seen from both graphs there are clearly some improvements to the response times.

Another graph I investigated was the ‘Statistical Aggregate Report’. This graph also shows improvements in response times and throughput after the fix:

Before

After

However, the report clearly shows that the response times increases with load whilst the throughput decreases with load, this indicates that the system doesn’t scale very well as it’s. So next step was to try load balancing.

Performance Testing with Apache and Tomcat Load Balancing

Ok, next step was to try to load-balance two Tomcat instances using Apache’s mod_proxy, this was tried on a Pentium 4 2.4GHz with 1.5GB RAM running Fedora Core Linux. And the results were promising as the Throughput is now stable, however response times were still increasing with the load profile:

Another thread Dumb at peak load have shown that the high response times was due to the blocking cause by the limited number of database connections in the database connections pool. With 50 Active users the database connections pool was set to the maximum of only 10 database connections.

Next step is to increase the database connections pool size and to retest.

To be continued…

How do we implement the User Interface to edit a list of complex objects, some of these objects might have a list of other objects as well?, there are a number of techniques to implement this, I present three of them below. You need to assess the usability and ease of implementation for each one before choosing a technique for your UI.

The easiest single view technique

I believe this is the easier and the technique that provide the least rich user experience.

Have a drop down list with an identifier for all the object with a new button next to it. When the user selects an item from the list an area is displayed with the fields populated with the object properties including a Save and Delete buttons. Clicking the Add New button results in the area displayed with blank fields

Example 1:

A drop down list with button to create a new record

Clicking the ‘New’ button displays empty fields

Selecting an existing record from the drop down list shows the fields prepopulated to provide the ability to edit the record

Example 2:

This example shows a more complex UI using the Dojo Toolkit to edit a collection of objects with each object having another collection of objects:

Selecting a row from the multiple select shows the second level of editable objects

The rich multi-view technique

This technique is highly effective in a MDI (Multiple Document Interface) view. But can also be used in a classical Web view where each view is displayed on a separate Web page. This technique is highly effective when editing complex object graphs specially when a new view is launched to edit or add a new object as shown in example 2 below.

Display all the details in a grid with new, save and delete buttons. The Grid is readonly. Selecting a record populates a form on the side or below the grid or pops a window with the details, after editing the user can click save. Clicking add new can either launch a window with a blank form or show a blank form to be edited

Example 1:

The example below shows a form binding to a grid, when the user makes a selection from the grid the form is pre-populated with the selected grid record. The form could also be used to add new records to the grid

Example 2:

The example below shows a grid with a number of buttons to edit/delete and create a new record

Clicking the edit button launches a new window to edit the record details

Clicking the new button launches a new window to add a new record

The rich single view technique

Have a fully editable grid with save, new and delete buttons. All operations are performed on the Grid. I believe this technique provides a very rich user experience if it utilizes Ajax to update the data once the user has finished updating the records. The view can also fit within a single Web page view. The technique may only useful when editing objects that have simple properties, but not objects that have a collection of other objects that also need to be editable.

Example 1:

Example 2:

WebEx is a bit of an overkill for casual use plus their pay-per-use tariff is also expensive. An alternative to WebEx was to use TightVNC, but again it was too much hassle opening the firewall port for the VNC traffic.

Using built-in Windows applications such NetMeeting and Windows Meeting Space was a problem as NetMeeting is missing in Vista and Windows Meeting Space only works in Vista. So I think this new feature in Skype will be very handy and very helpful for academics and small businesses that require screen sharing for free and with minimum setup.

How to use it?

From the Call menu select the Share Your Screen option, which presents two options Share Full Screen or Share Selection. The Share Full Screen option provides the ability to share any of the screens currently connected. Good for sharing a secondary monitor for a presentation

After selecting the Share Selection option, a rectangle is displayed on the screen, which can be resized

Click the start screen sharing button and the rectangle border becomes red indicating that the red area is now shared with the participant in the conversation

This is the view that other participants see, they can click the full screen option for better resolution

The web server I’ve considered was Apache web server and the application server was Tomcat Servlet container. At the time a had very limited knowledge on the whole concept of an application server and firewalls.

Another diagram I did a few years later after understanding the concept of clustering and SSL

[omerio@future TestProject]$ svn import -m “Initial Import” /tmp/TestProject http://salam/repos/TestProject
Authentication realm: <http://salam:80> Authorization Realm
Password for ‘omerio’:
svn: applying log message to /repos/!svn/wbl/c0e1dee1-2e96-432e-a904-054d922233d8/0: 501 Method Not Implemented (http://salam)

I checked my svn setup and everything was fine, even a Google search didn’t help. Luckily I remembered that I have mod_security with Apache, a quick check revealed  that mod_security was blocking subversion’s http requests as seen from the mod_security logs

[Sat Sep 26 13:13:59 2009] [error] [client 192.168.1.5] ModSecurity: Access denied with code 501 (phase 2). Match of “rx (?:^(?:application\\\\/x-www-form-urlencoded(?:;(?:\\\\s?charset\\\\s?=\\\\s?[\\\\w\\\\d\\\\-]{1,18})?)??$|multipart/form-data;)|text/xml)” against “REQUEST_HEADERS:Content-Type” required. [file “/etc/httpd/modsecurity.d/modsecurity_crs_30_http_policy.conf”] [line “69″] [id “960010″] [msg “Request content type is not allowed by policy”] [severity “WARNING”] [tag “POLICY/ENCODING_NOT_ALLOWED”] [hostname “salam”] [uri “/repos/!svn/wbl/909077f2-7e84-4b7d-8e8a-8cbc35fc1ef0/0″] [unique_id “Sr4Fh38AAAEAABSMGt4AAAAA”]

I checked Google and found this page which was very useful, so I added a number of mod_security exceptions to the subversion.conf Apache configuration file, Although I had to commit a few projects to get a list of the violated mod_security rules. I think if I get any mod_security issues with svn I will disable the security engine for svn

<Location /repos>

   DAV svn

   SVNPath /var/opt/apps/svn/repos

   AuthType Basic

   AuthName "Authorization Realm"

   AuthUserFile /var/opt/apps/passwd/passwords

   Require valid-user

   <IfModule mod_security2.c>

        SecRuleRemoveByID 960032

        SecRuleRemoveByID 960015

        SecRuleRemoveByID 960904

        SecRuleRemoveByID 960010

        SecRuleRemoveByID 960035

        SecRule REQUEST_METHOD "^(PROPFIND|PROPPATCH)$" allow

        SecRule REQUEST_METHOD "^(REPORT|OPTIONS)$" allow

        SecRule REQUEST_METHOD "^(MKACTIVITY|CHECKOUT)$" allow

        SecRule REQUEST_METHOD "^(PUT|DELETE|MERGE)$" allow

        SecRule REQUEST_METHOD "^(MKCOL)$" allow

   </IfModule>

</Location>

Sep 23, 2009 7:57:15 AM org.apache.catalina.core.StandardContext start

SEVERE: Error filterStart

Sep 23, 2009 7:57:15 AM org.apache.catalina.core.StandardContext start

SEVERE: Context [/OfficeMAWeb] startup failed due to previous errors

This is usually due to conflicting struts 2 jar files in the webapp lib folder, the error will be explained in details in the localhost<date>.log, in my case I had both struts2-core-2.1.6.jar and struts2-core-2.0.12.jar in the lib folder

INFO: Initializing Spring root WebApplicationContext

Sep 23, 2009 7:51:09 AM org.apache.catalina.core.StandardContext filterStart

SEVERE: Exception starting filter struts2

Unable to load bean: type: class:com.opensymphony.xwork2.ObjectFactory - bean - 

jar:file:/home/omerio/apache-tomcat-6.0.20/webapps/OfficeMAWeb/WEB-INF/lib/struts2-core-2.1.6.jar!/

struts-default.xml:29:72

        at com.opensymphony.xwork2.config.providers.XmlConfigurationProvider.register(XmlConfigurationProvider.java:208)

        at org.apache.struts2.config.StrutsXmlConfigurationProvider.register(StrutsXmlConfigurationProvider.java:101)

        at com.opensymphony.xwork2.config.impl.DefaultConfiguration.reload(DefaultConfiguration.java:131)

        at com.opensymphony.xwork2.config.ConfigurationManager.getConfiguration(ConfigurationManager.java:52)

        at org.apache.struts2.dispatcher.Dispatcher.init_PreloadConfiguration(Dispatcher.java:395)

        at org.apache.struts2.dispatcher.Dispatcher.init(Dispatcher.java:452)

        at org.apache.struts2.dispatcher.FilterDispatcher.init(FilterDispatcher.java:203)

        at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)

        at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397)

        at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:108)

        at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3800)

        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4450)

        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)

        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)

        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)

        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:850)

        at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:724)

        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:493)

        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)

        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)

        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)

        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)

        at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)

        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)

        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)

        at org.apache.catalina.core.StandardService.start(StandardService.java:516)

        at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)

        at org.apache.catalina.startup.Catalina.start(Catalina.java:583)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)

        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)

Caused by: Bean type class com.opensymphony.xwork2.ObjectFactory with the name

 xwork has already been loaded by bean - jar:file:/home/omerio/apache-tomcat-6.0.20/

webapps/OfficeMAWeb/WEB-INF/lib/struts2-cor

e-2.0.12.jar!/struts-default.xml:30:72 - bean - jar:file:/home/omerio/apache-tomcat-6.0.20

/webapps/OfficeMAWeb/WEB-INF/lib/struts2-core-2.1.6.jar!/struts-default.xml:29:72

        at com.opensymphony.xwork2.config.providers.XmlConfigurationProvider.register

(XmlConfigurationProvider.java:193)

        ... 33 more

Sep 23, 2009 7:51:09 AM org.apache.catalina.core.ApplicationContext log

INFO: Closing Spring root WebApplicationContext

Eventually I ended up removing persistence.xml and orm.xml from the jar file and adding the JPA annotated classes in the jar file to my Webapp persistence.xml. It worked, but Eclipse JPA claims it can’t resolve the class and shows an error on persistence.xml.

The Webapp works fine so in the meantime I won’t bother about the Eclipse error, but obviously one day I will be Googling for a solution.

HOWTO: IPCop-OpenVPN: http://thinkhole.org/wp/2006/03/28/ipcop-openvpn-howto/

HOWTO:Roadwarrior:http://www.zerina.de/zerina/?q=documentation/howto-roadwarrior

Creating the server certificate took ages in my IPCop machine, bearing in mind it’s a Pentium MMX processor 233MHz :-). The certificate generation doesn’t use that much memory, but it’s CPU intensive.

I first thought it would be better to connect using the Vista wizard, but as usual username & password authentication was easy to setup, however the client certificate setup was a fiddly one. I opted to save my time, abandon Vista Wizard and install OpenVPN client.

After installing the OpenVPN client for Windows (the recent openvpn-2.1_rc15 include OpenVPN GUI as well) remember to review the README file as it includes important information for Vista users.

Due to elevation restrictions in Vista, OpenVPN client will fail to add routes to the routing table if not run as an Administrator. Simply right click on the OpenVPN GUI icon and select run as Administrator. Check the status log for any errors.

I managed to test my VPN connection using my mobile broadband after disabling my LAN wireless connection.

I have always been a fan of Red Hat Linux, starting from RHL8,9 and I was there when Red Hat decided to split it into the Fedora project. And I was there when Fedora Core 1 came out, I tried it and tried Fedora Core 3, I adopted Fedora 5 as my server’s OS and I went to tightly secure Fedora 8 as part of my uni security course. I always loved Red Hat and loved their Linux. In-fact I have never tried any other linux distro other than RHL and Fedora (Although I did install IPCop, a linux firewall distro).I downloaded the Fedora 10 DVD image and as usual for some reason it was corrupt and ended up wasting a couple of hours on the installation, which I then abandoned and decided to download the netintall image which worked perfectly, but I had to leave it overnight to download the required packages.

Amazingly there was a large number of Educational programs and kids games, plus hundreds of other packages of useful stuff. My daughter were so happy to play the games, so I left them on the console and I used vnc from my laptop to have my own X desktop and finish off the installation of other stuff. It just makes realize how powerful Linux is. Anyway as usual with any of my Linux installations I had to add the static routes to manage my sub-nets, I ran the commands then put them in /etc/rc.d/rc.local

route add -net 192.168.3.0 netmask 255.255.255.0 gw 192.168.1.5 dev eth0

route add -net 192.168.3.0 netmask 255.255.255.0 gw 192.168.1.5 dev eth1

All in all, performance was good, the installation was easy, all the hardware was recognised automatically, the only problem I encountered was my fault due to my corrupt image, but other than that it an easy install. Definitely Linux has come a long way and specially Fedora and at version 10 it looks awesome.

Using Amazon’s pricing calculator I tried to enter the same specifications offered by a PoundHost A2 dedicated server and to my shock Amazon EC2 monthly pricing was three times that of PoundHost A2 dedicated server. So what I don’t understand is how would a startup be able to afford using a more expensive service such as EC2?, may be it’s the buzz word ‘Cloud’ and lots of cash coming out of VC funding rounds!

Personally I think Amazon is still got a great deal to do in regards to pricing if they want to compete with the dedicated server hosting market.