These are a few of my favorite apps…

I’m sure everyone’s got ’em. When you get past the facebook an twitter clients, what are the iOS apps you really find useful? Here’s a bunch of mine. What’s yours?

Air Video – This is a really cool app and accompanying server to let you play your own videos, anywhere. No iCloud necessary! Dup your DVD collection and watch it where ever an whenever you want. I’ve got about 2Mbit outbound bandwidth which is more than enough for nice looking video on any iOS device.

Xfinity TV – for as much as we all hate Comcast, this app fills some glaring voids in their on-screen UI. Search listings easier. Better organization to on-demand offerings and even control your TV from the app.

IM+ Pro, Touch Term, Pocket Cloud – this trio of apps is a pretty good set to have for any system administrator, giving you instant messaging, SSH, VNC and RDP clients.

Wikinvest, Mint – both great tools to manage your investments and personal finances. Tie in pretty well with their respective sites.

Mobile Mouse – another client/server pair. Gives you a virtual keyboard & mouse so you can control your Windows an Mac OS systems.

Reeder – nicely packaged Google Reader enabled feed reader. This replaces Mobile RSS which, up until iOS 5 was my reader of choice.

Push Mail – great app/service to allow you to send Notifications to your phone via email. You can configure different profiles for different messages to display different email fields in the Notifications.

IProxy – happen to still have an unlimited data plan and a developer device? Use iProxy instead of enabling full fledged tethering.

Flipbook – yeah the UI I’d kind of gimmicky and hard-cores will probably get their news elsewhere but it’s a nice read for standing in the grocery line.

Advertisements

MySQL user@localhost and user@%

Its always bugged me that I have to create two user accounts in MySQL, one with a wildcard host (%) and another for localhost.

Finally investigated it enough to understand why. Briefly, three things are going on.

  1. First, MySQL takes a non-wildcard host as precedence when looking for user accounts
  2. The mysql_install_db script create an anonymous user account ”@’localhost’
  3. The anonymous use account takes precedence when the hostname matches.

So the net effect is that when you connect locally, in general MySQL will use the anonymous account to authenticate to, no matter what you specify as a username, if the only account that matches the username also has a wildcard host field.

This is explained in the MySQL reference manual:

It is necessary to have both accounts for monty to be able to connect from anywhere as monty. Without the localhost account, the anonymous-user account for localhost that is created by mysql_install_db would take precedence when monty connects from the local host. As a result, monty would be treated as an anonymous user. The reason for this is that the anonymous-user account has a more specific Host column value than the 'monty'@'%' account and thus comes earlier in the user table sort order.

I’m not quite sure why an anonymous user is created by default, some documentation suggests that on Windows, its used as the ‘root’ account.

Interestingly, although you need both user@localhost and user@%, you don’t also need to grant privileges to both users. It appears to be sufficient to grant privileges to user@% and those privileges will also be used for user@localhost (this was another source of confusion for me.)

If it bugs you so much, there are other options

  • remove the anonynous account(s) entirely. I’ve seen references in a couple of places suggesting this is an okay thing to do
    delete from mysql.user where user = ''
  • rename the anonymous account(s). If deleting them seems a little too drastic
    update mysql.user set user = 'visitor' where user = ''

I’m not sure which option I like, but now I finally understand MySQL authentication better.

NetBeans code templates for Builder Pattern

Here’s a couple of handy code templates (short cuts) for developing classes that use the Builder pattern.

This first one creates the Builder inner class itself. I’ve used the abbreviation “pscB” for this (Public Static Class Builder) –

public static class Builder {
    public Builder() {
        
    }
    
    public ${Type} build() {
        ${Type} result = new ${Type}();
        
        return result;
    }
    
    ${cursor}
}

And this one will simplify creating a “setter” field within the Builder, I use the abbreviation “psBB” for this –

public Builder ${field}(${Type} ${field}) {
    this.${field} = ${field};    
    return this;
}

IOps calculations

So I got a disk array and I want to know what sort of IOps workload it can support.

Articles like IOps? provide a pretty standard formula I have seen before, but only answers the question “Given a required workload IOps for an application, how big does the array need to be to support it?” This equation is

Array IOPS = (Workload IOPS * Read Workload Percent) + (Workload IOPS * Write Workload Percent * RAID IO Penalty)

However, sometimes you want to answer the opposite question “Given a total number of IOps for an array, how much workload IOps can it support?”

The answer is really just to solve for Workload IOPS in the above equation. To save you the math,

Workload IOPS = Array IOPS / (Read Workload Percent + Write Workload Percent * RAID IO Penalty)

This could have been pretty handy a couple of years ago when I was trying to build a private cloud, to help me understand how much workload the arrays we used would support  – and therefore figure out how many virtual machines we could run.

Henplus with Java-Readline for OS X

UPDATE: The procedure below is not entirely correct. It will work, but its not the most straight forward, and it probably won’t get you what you are really looking for. I have written a new procedure for building Java-Readline and GNU readline for OS X. Please use these instructions instead.


 

So, I found it time to revisit an old friend, HenPlus since I’ve been mucking around in a few databases lately.

The wrinkle this time, OS X. The main problem being the JNI interface to the GNU Readline library Java-Readline does not compile all that well on OS X.

Doing a bit of searching I found a few articles describing the steps to compile it, but they are slightly out of date for OS X 10.7 Lion, so I figured I’d put in my two cents.

For starters, Nathan Toone’s article Java-Readline on Mac OS X Update gets pretty close, but things have changed since he wrote it in 2006. So here’s my redux based on it.

Note that in my case, I did not care about building a universal binary (if that’s even possible on Lion) so I didn’t have to make as many changes as he did.

Tweaking Java-Readline

Unpackage.

In the source root, edit Makefile and make the following changes:

  • Below the line # Operating system dependent
    Add JAVA_HOME = `/usr/libexec/java_home`
  • Clear out the JAVANATINC variable
  • Change the JAVAINCLUDE variable
    to /System/Library/Frameworks/JavaVM.framework/Headers
    (This is needed to find jni.h)
  • Change LD_LIBRARY_PATH to be DYLD_LIBRARY_PATH

Next, edit src/native/Makefileand change the following:

  • Clear out the LIBPATH variable
  • Change the CFLAGS to -fno-common -DMAC_OS
  • Change $(CC) -shared (OBJECTS) $(LIBPATH) $($(TG)_LIBS) -o $@ to
    $(CC) -bundle -flat_namespace $(OBJECTS) $(LIBPATH) $($(TG)_LIBS) -o $@
  • Change the JavaReadline_LIBS variable to be -lreadline -ltermcap

Now, you need to edit src/native/org_gnu_readline_Readline.cand make the following changes:

  • On lines 98, 114, 213, and 224,
    change #ifdef JavaReadline to #if defined JavaReadline && !defined MAC_OS
  • On lines 216, 235, and 475,
    change #ifdef JavaEditline to #if defined JavaEditline || defined MAC_OS

Now, you can run make, and you will end up with libJavaReadline.so and libreadline-java.jar.

Rename libJavaReadline.so to libJavaReadline.jnilib.

I’ve made a patch script of all the changes to save anyone some effort in editing all the files by hand. Simply run

  • patch -p1 < libreadline-java-0.8.0-osx.patch

HenPlus Notes

Once you have compiled Java-Readline, you can compile HenPlus as described. A couple of things to note though

  • Copy libreadline-java.jar from Java-Readline into <INSTALLDIR>/share/henplus
  • Make <INSTALLDIR>/lib and copy libJavaReadline.jnilib into it
  • Change all occurrences of LD_LIBRARY_PATH in bin/henplus to DYLD_LIBRARY_PATH

And from there you should be set.

Sending email as multiple identities with Gmail & Thunderbird

I just figured this one out, and before I forget it.

I’ve been a user of Thunderbird’s Multiple Identities feature for a while. It allows you to send mail from a single account using multiple identities. So, for example, I have my personal email address for the majority of my email, but I have a second address which is really an alias for my wife and I which we give to day care, schools, etc. And when communicating with said entities I want the email to look as if it came from the alias.

I’m not sure when it changed, but recently even though I have set everything up the same way in Thunderbird, when the email was delivered to the receiver, it would show up as my primary address. Not good.

I immediately suspected the recent releases of Thunderbird, 5.0 & 6.0, but a few searches and I found nothing.

The only other change was switching over to Google Apps to host my personal domain email, I had done that a while ago though. But still poked around anyways.

And wouldn’t you know I found two settings, either one or both of them fixed the problem.

These are both in Mail Settings -> Accounts (note Mail Settings is under the Gear icon in the upper right)

  • First I added the email alias to my account by clicking on Add another email address you own link.
  • Second I set When replying to a message to Reply from the same address the message was sent to.

Not sure which of these fixed the problem, I suspect the second, but either way, now the receiver correctly sees the alias when I choose  to send mail as that identity.

SSL Termination, Load Balancers & Java

So you got this killer Java web app you are developing, or you are deploying some super cool off the shelf application, and you decide to use some sort of SSL Accelerator to offload your HTTPS traffic, and it looks something like this:


Only things aren’t going so well. Maybe all the generated URLs for HTTPS connections are coming out as plain HTTP. Or your web app is attempting something clever and ntify when specific requests are plain HTTP and redirect them over to HTTPS, however even after redirection the app still tries to redirect again.

What Gives?

As you have probably guessed, the introduction of the SSL Accelerator has caused a problem because now all the incoming requests to your web app are coming in as plain HTTP, because HTTPS was terminated upstream somewhere and so your application server never knows about it.  In Java web applications, this generally manifests itself as HttpServletRequest.isSecure() always returning false, wreaking havoc in its wake. The Servlet machinery makes lots of decisions based on the value of isSecure(). All the way down to when you are defining security-constraint elements in web.xml and setting transport-guarantee to CONFIDENTIAL.

So What’s the Fix?

I’ve encountered this quite often, with various SSL Accelerators, Load Balancers and App & Web Servers. In general, the idea is to maintain separate paths for the requests all the way through to the app servers, like so:


In doing so, you can typically configure the back end web or app server to correctly identify the originating scheme of the request (HTTP or HTTPS).

Any Examples?

The Easy – Plain Ol’ Tomcat Backend.

By far the easiest container to work with is Apache Tomcat. The options available in a Connector clearly had this type of configuration in mind. Specifically these are

  • scheme – this tells Tomcat which protocol to use to create links, either http or https.
  • proxyPort – this tells Tomcat what to use as the real originating port, instead of the port the connector is listening on.
  • proxyHost – similarly, this tells Tomcat what to use as the real host name, instead of the host Tomcat is running on.
  • secure – this tells Tomcat whether the Connector is secure or not.
  • SSLEnabled – whether to configure the connector for SSL/TLS or simply Plain HTTP.

So, in the case where you are only using Apache Tomcat in your platform,

The correct configuration can be accomplished by creating two Connectors, one for port 8080 and one for port 8081. These would look like

<!--
     Define the connector on port 8080 to handle
     originating plain HTTP requests. Nothing
     special here. Make sure to set proxyPort
     and proxyHost to the appropriate values
     for your web site.
-->
<Connector port="8080" protocol="HTTP/1.1"
           maxThreads="150" clientAuth="false"
           SSLEnabled="false"
           scheme="http" secure="false"
           proxyPort="80" proxyHost="my.website.com"
/>

<!--
     Define the connector on port 8080 to handle
     originating HTTPS requests. Here we
     set scheme to https and secure to true. Tomcat
     will still serve this as plain HTTP because
     SSLEnabled is set to false.
-->
<Connector port="8081" protocol="HTTP/1.1"
           maxThreads="150" clientAuth="false"
           SSLEnabled="false"
           scheme="https" secure="true"
           proxyPort="443" proxyHost="my.website.com"
/>

And then, of course, configure your load balancer (and SSL Accelerator) to direct traffic to the appropriate ports.

A Little Harder – Tomcat with Apache Httpd

Introducing Apache Httpd into the mix really all that much harder than using simply Tomcat. Here’s the desired traffic pattern.

You may notice that HTTP protocol stops at the Httpd and then from Httpd to Tomcat the protocol switches to AJP. And then there’s the two streams coming into Httpd, but lets tackle Tomcat first. Its actually not very different at all from straight Tomcat.  The only difference is switching to AJP/1.3 for the protocol.

<Connector port="8009" protocol="AJP/1.3"
           maxThreads="150" clientAuth="false"
           SSLEnabled="false"
           scheme="http" secure="false"
           proxyPort="80" proxyHost="my.website.com"
/>

<Connector port="8010" protocol="AJP/1.3"
           maxThreads="150" clientAuth="false"
           SSLEnabled="false"
           scheme="https" secure="true"
           proxyPort="443" proxyHost="my.website.com"
/>

Now, for Httpd. This is solved by setting up a separate virtual servers for each port, and passing them off to the appropriate Tomcat Connector using mod_jk and separate worker pools for each Tomcat Connector.

JkWorkersFile /etc/httpd/conf/workers.properties

<VirtualHost *:80>
    DocumentRoot /var/www/html
    ServerName my.website.com

    JkMount /webapp/* tomcat-8009

</VirtualHost>

<VirtualHost *:81>
    DocumentRoot /var/www/html
    ServerName my.website.com

    JkMount /webapp/* tomcat-8010

</VirtualHost>

And now the workers.properties file.

worker.list= tomcat-8009, tomcat-8010

worker.tomcat-8009.type=ajp13
tomcat.tomcat-8009.host=server1
tomcat.tomcat-8009.port=8009

worker.tomcat-8010.type=ajp13
tomcat.tomcat-8010.host=server1
tomcat.tomcat-8010.port=8010

And that’s it. The one caveat here, is that although originating from HTTPS and plain-HTTP follow different paths even through Httpd, Httpd doesn’t really know that. So if you are doing anything interesting other than serving static content from Httpd, you may have an issue, for example if you are running PHP or something else that may need to care, you may be out of luck.

The Difficult – Less Configurable Application Servers

The Tomcat team as really taken a lot of time to provide a highly configurable platform, but what if this is not your target deployment platform, and you are faced with attempting to do SSL Termination on a platform that isn’t so helpful. One example that I’ve got a bunch of experience with is what was called Sun Java System Web Server and now Oracle iPlanet Web Server. Although I did enjoy working with it, the platform did not provide such an easy method of integrating an external SSL Accelerator.

But all is not lost. Fortunately Java Servlets version 2.3 and newer provide some facilities that can help. Servlets 2.3 provide two interesting features, Filters and Wrappers.

Filters provide a method for you to do some pre-processing of requests before your Servlets, JSP and anything else is invoked (they also can do post-processing but we don’t need that here.)

Wrappers provide a method for you enclose a Request or Response in your own object.

The two, together can help you fake the Java machinery into setting HttpServletRequest.isSecure() and HttpServletRequest.getScheme() into returning the correct values depending on the original protocol.

Lets look at creating a Wrapper first. Its as simple as

public class FakeSecurityRequestWrapper extends HttpServletRequestWrapper {
    public FakeSecurityRequestWrapper(HttpServletRequest request) {
        super(request);
    }

    public boolean isSecure() {
        return true;
    }

    public String getScheme() {
        return "https";
    }
}

Pretty trivial. Now to create a Filter to inject the wrapped Request into the machinery.

public class FakeSecurityFilter implements Filter {

    public void doFilter(ServletRequest req, ServletResponse res,
        FilterChain chain) throws IOException, ServletException {

        FakeSecurityRequestWrapper fakereq =
            new FakeSecurityRequestWrapper(req);
        chain.doFilter(fakereq, res);
    }
}

Now just plumb this up in your Web App’s web.xml as the first filter in the chain, and you are there. Well, most of the way there. there’s the opposite problem now. Every request, no matter what the originating protocol, appear to be HTTPS. In a mixed protocol environment, that is still a problem.

No worries, we can do it! Yes, with a little more tweaking, it should be possible to support a mixed protocol environment.

And its with a little trick from the load balancer or SSL Accelerator. Most of them that I have run into allow you to modify the request as its passing through the appliance. To the idea is to insert a Header into the HTTP request for request that have gone through the SSL Accelerator. A typical flow would look something like this.


I won’t go into the configuration on the SSL Accelerator/Load Balancer end, but lets look at enhancing the Filter to handle mixed protocols. In this example, we’ll assume the Accelerator/LB sets the header X-Original-Protocol to https for requests coming through it. The doFilter() method becomes

    public void doFilter(ServletRequest req, ServletResponse res,
        FilterChain chain) throws IOException, ServletException {

        String origProto = req.getHeader("X-Original-Protocol");
        if (origProto != null && origProto.equals("https")) {
            FakeSecurityRequestWrapper fakereq =
                new FakeSecurityRequestWrapper(req);
            chain.doFilter(fakereq, res);
        } else {
            chain.doFilter(req, res);
        }
    }

Voila. Now the Java Servlet machinery should be handling a mixed protocol environment just fine. This technique should work with just about any web container, be it WebLogic, JBoss, SJSWS/IWS or GlassFish.

Its arguable whether this method is more straight forward than the other previous methods, but it does have it down sides, which for me, make it less desirable. It does require a little programming, and needs actively integrated into each web app you are deploying and I don’t like having to muck with 3rd party web.xml files if I don’t need to, and for that I leave this as a last resort option.

I’m sure there are other techniques for working with SSL Accelerators, these are just the methods that worked for me so far. Hope others find them useful.