BukGet Infrastructure Expansion

Sun 29 June 2014 Posted by Steven McGrath Category BukGet

tags: python  bukget  minecraft  Comments

 

NOTE: Cross-posted from BukGet's news page.

In the past we have run into issues with the API simply keeping up the demand, causing us to expand out to a second server some years ago to help distribute the load. More recently however, the architecture that we have been priding ourselves in being simple and efficient bit us in the butt when the US server went off-line for an extended period of time. Unfortunately for us, this was beyond our control, as the hosting provider that was donating us the gear ran into problems and their entire facility went off-line. We scrambled to being up a temporary master so that generations could continue as normal, however we had to develop a better model for handling this.

Our initial thought was to try to take advantage of the situation and get the community to help us raise the capitol to run our own gear, our way. Unfortunately reality set in pretty quickly when our rally cries were either deleted on Reddit, or generally ignored. A few companies however did respond to offer us some more donated gear and gratis hosting in order to keep us up and running. As it seems that this model is the only model that the community wants to support, we eventually conceited defeat on trying to get our own gear and started communicating with some of the various hosting providers that had contacted us.

With this in mind, we started to engage several companies and then quickly narrowed the list down from there. We wanted to still keep the overall environment small so that it's manageable by our 2-man (lets be honest, David does most of the ops stuff now) operations team. We also made some small, but fundamental changes to how we will be handling the data, generations, etc. as well. As a result, we went from 2 servers to 6, and we will also be looking into making some other changes down the road to help us react quickly and effectively to any future changes.

The first change we did was make the decision to have dedicated generation servers. In the past, all generation was done on the Dallas API server, and while this never cause any issues, we were putting too many eggs in one basket. We also figured that is 1 generation server is good, it might make sense to have a backup server in-case the first one goes down for any reason (like in-case, you know, our provider goes down).

The second change is that we now will have 2 servers in each geographic region. Each server will also be hosted by a different provider. This means if something goes down, that whole region wont be be (hopefully) significantly impacted.

Thirdly, we decided to expand out the MongoDB instances a touch, making sure that the API servers are all identical (and now slaved) to the current active master generator.

Lastly, while the city-based naming convention worked well for us, it often created some confusion. So we are switching up to a state/country model with each region also rolled up into it's respective regional record. This means that to a large degree, we have added some more granularity to how we are referencing the our systems, and also help developers when troubleshooting issues with their code.

In general our goal here is to try to make the BukGet platform generally more stable for everyone and provide the needed resources to grow. We have already talked internally about starting to support more servers than just Bukkit, however some of these changes were needed first to really bring everything up to speed before we could start working down those paths.

Comments


DoFler a hit at CircleCityCon

Sun 29 June 2014 Posted by Steven McGrath Category Misc

tags: python  dofler  circlecitycon  Comments

 

DoFler was a rousing success earlier this month at CircleCityCon as quite a few people enjoyed watching what was going on with the conference wireless, trolling the network to see what would show up, and having a lot of conversation about what it's doing and what can be done ...

BukGet API3 gets Plugin & File IDs.

Mon 27 January 2014 Posted by Steven McGrath Category BukGet

tags: python  bukget  minecraft  Comments

 

BukGet has been updated with the curse_id and file_id attributes for plugins and versions respectively. These ids are considered non-changing within the Curse systems and are there for plugin devs and plugin management devs to wire into for something more concrete than the slug names.

What intended next is to ...

BukGet gets a new Dallas Server

Mon 06 January 2014 Posted by Steven McGrath Category BukGet

tags: python  bukget  minecraft  api  Comments

 

After the fallout of the ShardHost debacle, generously stepped up to fill the void. To be honest a lot of this had happened back in November, however due to my day job, the holidays, and some general laziness (I would like to play the occasional game as well you know ...

Dofler 0.4.1 Released

Mon 06 January 2014 Posted by Steven McGrath Category Misc

tags: python  dofler  conferences  wallofewes  Comments

 

Dofler Version 0.4.1 has been released. I feel confident that this version is stable enough for general use. The whole WebUI has been completely overhauled to make setting the system up easier to use and to open up Dofler installations beyond x86 hardware. ALso all of the dofler-lite ...

Page 1 / 6 »