Where We Are Right Now

Some issues were found with the PaaS2 results, regarding received errors. When planning this experiment, we went ahead without accounting for the fact that a client machine's speed affects how much uptime a cloud app experiences. The implication of this is that services with a limit on an app's uptime will experience a client making requests differently depending on the client's speed.

Say you have two clients talking to two instances of the same app. Each instance has 4 hours of allotted uptime per day, and once that uptime is exceeded any client sending requests to these apps will receive only 500 errors in response. One of your clients causes 20 minutes of uptime per run of a program, and the other causes 30 minutes per run of an identical program; this is due to hardware differences between the two clients. If you are required to run this program 9 times per day per app/client pair, and each client is constrained to one app, the app running 20 minutes per client program will not exceed its allotted time, but the app running 30 minutes will.

This is an oversimplification of our problem. First, we have to factor in the fact that we don't know how much time one request to an app instance will eat up on the server end. Conditions on the server end may change, and the vendor's method for timing uptime is unknown and seems to have other factors beyond straight app uptime involved. In addition, we have to take into account travel time to the server, which is affected by network conditions in way we cannot predict. This adds to our client uptime, in turn affecting our app uptime.

There are a few morals here: "careful planning can still surprise you with bumps," "networking is complex," etc. The biggest, though, is that we have to restart data collection on PaaS2 for 2 of our 3 clients.

I restarted yesterday, and the results are good, but PaaS2's dashboard did show error replies in some of the runs. Apparently we haven't hit that sweet spot yet where they occur so minimally that we don't have to worry about them.