Great API! Are there any concurrency limits? I’d like to make some calls in parallel for a small project using Spark. I’m afriad I’ll tank the system however. Any recommendations?
Thank you! I spent the last several hours testing concurrency. With the hosting service I am using, it did end up crashing when I sent over a handful of parallel requests via Apache JMeter. I tried implementing clustering, which allows you to distribute the processes from each request amongst different threads. However, this is only really helpful if the tasks are CPU-intensive. The response times from this API are mainly due to network delays from web scraping, so when I tried clustering, it actually worsened response times. It also ended up using more memory than this hosting service allows.
If you are going to do parallel requests, I’d recommend only a few at a time.
Also, I just fixed a couple bugs and you should now be able to use the hourly endpoint. The lift elevations in the “basic info” are also now usable.
Thank you! I spent the last several hours testing concurrency. With the hosting service I am using, it did end up crashing when I sent over a handful of parallel requests via Apache JMeter. I tried implementing clustering, which allows you to distribute the processes from each request amongst different threads. However, this is only really helpful if the tasks are CPU-intensive. The response times from this API are mainly due to network delays from web scraping, so when I tried clustering, it actually worsened response times. It also ended up using more memory than this hosting service allows.
If you are going to do parallel requests, I’d recommend only a few at a time.
Also, I just fixed a couple bugs and you should now be able to use the hourly endpoint. The lift elevations in the “basic info” are also now usable.