Over the past few weeks, I've been conducting tests to determine Heroku's suitability as a platform for dealing with large amounts of complex computations. The results of some of these tests are posted below. The textual results represent various aspects of the running time of the program, and the videos are aggregations of the best-fit member of each generation and the members of the last generation for each population.
The time results tell me that the tests are running very quickly, fantastically quickly. The test of 125 populations ran in around an hour and seven minutes--that's 500,000 calculations done in just over an hour. However, looking at the results of the tests tells me that the reason it runs so quickly is that the tests are timing out. When this happens, they use a default value that was set up to indicate a timeout. The results should be in the hundreds, and instead they are consistently under 5, which mans there is a problem.
There are a few working hypotheses as why these results are occurring. After looking at them with Dr. Remy, he confirmed that there is definitely an issue, but that the results don't look like the expected output for any of his built-in networking error values. This could indicate that the GA is receiving random values from the app. If so, there could be a problem with the GA or app, Heroku may have been experiencing some kind of technical difficulties at the time, we may have found a bound on the capabilities of one dyno, the network may have had issues while running--the list goes on. In any case we're investigating the results.
Populations: 50
Individuals/Pop: 50
Generations: 80
Start: 1402348187.4
End: 1402351825.8
Test Duration: 3638.39999986s = 1.01066666663hr
Average Run: 3556.59600002s = 0.987943333338hr
Longest Run: 3601.20000005s = 1.00033333335hr
Shortest Run: 3466.79999995s = 0.962999999987hr
Populations: 75
Individuals/Pop: 50
Generations: 80
Start: 1402363852.5
End: 1402367892.0
Test Duration: 4039.5s = 1.12208333333hr
Average Run: 3846.72399996s = 1.06853444443hr
Longest Run: 3969.5s = 1.10263888889hr
Shortest Run: 3676.4000001s = 1.02122222225hr
Populations: 100
Individuals/Pop: 50
Generations: 80
Start: 1402379304.6
End: 1402383480.4
Test Duration: 4175.80000019s = 1.1599444445hr
Average Run: 3928.14699999s = 1.09115194444hr
Longest Run: 4114.9000001s = 1.1430277778hr
Shortest Run: 3733.4000001s = 1.03705555558hr
Populations: 125
Individuals/Pop: 50
Generations: 80
Start: 1402408981.8
End: 1402413277.3
Test Duration: 4295.5s = 1.19319444444hr
Average Run: 4106.75600002s = 1.14076555556hr
Longest Run: 4198.0999999s = 1.16613888886hr
Shortest Run: 3962.89999986s = 1.10080555552hr
No comments:
Post a Comment