Hi All, The "cloud market" when Apache Libcloud was conceived in 2010 is very different to how it looks today, some trends we are seeing
1- IaaS (our compute API) is one of many features in public clouds. Amazon, Azure and GCP have 100's of individual services now [1]. Text-to-speech, functions, automation, API gateways, it's impossible to keep up 2- Private Clouds have seen a continued decline but are still popular [2] 3- The advent of containers means it is now "easier" to deploy an application to multiple clouds 4- The big 3 public clouds, Amazon, Azure and Google make up most of the cloud market [3] if you compare Apache Libcloud downloads with boto (the native AWS Python client) downloads, it's a massive delta. Boto is in the top 10 most popular PyPi packages In terms of users, I've pulled 3 snapshots of PyPi downloads, January 2016, 2017 and 2018 [4] Annual downloads of Apache-Libcloud have seen a slight increase, but the 2016 - 2,778,687 2017 - 2,958,591 Python 2.7 represented 90% of users in 2017 and 64% in 2018. This is a huge drop. [4] *I would like to propose a drastic (depending on your perspective) plan to take Apache Libcloud through to 2020* *1. Focus on specific use cases* Lack of consistency between implementations of the base class means in practicality it's difficult to have abstracted applications. Apache Libcloud should (imo) come with a toolbox of utilities to both demonstrate and validate cloud abstraction use cases, such as: - Migrating storage objects from Cloud X to Cloud Y - Splitting an application workload across multiple clouds *2. Improve performance by adopting asyncio* In almost all use cases, Libcloud would benefit from non-blocking calls. Listing VM's requires multiple calls for the pages, uploading storage objects can be done in multiple futures, deleting DNS records would be better done in async. I'm suggesting we introduce a Python 3.5+ only API, move to requests-futures or aiohttp for the base HTTP client. Yes, *I am suggesting we drop Python 2 support in the future*. I've been researching how we could make this switch without breaking 64% of our users.. Pip now has a way to choose versions based on Python runtimes https://hackernoon.com/phasing-out-python-runtimes-gracefully-956f112f33c4 We could have apache-libcloud 3+ for Python 3.5 users and then maintain 2.3 patches for Python 2.7 and 3.4 users. I think we can come up with a way of continuing to maintain the existing code base for 2.7 users but move forward with a new API for async and Python 3.5+ users. The downloads [4] show that Python 2.7 is still the majority of users but this is declining quickly and by 2020 the tables will have turned. We need to be ready for that. *3. Change our positioning on dependencies of 3rd party packages* We aren't seeing enough community contribution to keep up with the rapid pace at which Microsoft, Amazon and Google are changing their APIs. Google have been contributing to their driver for years. We haven't seen that from either Amazon or Microsoft. Alibaba have contributed to theirs, many other cloud providers have contributed, but it seems to be after APIs are changed, not in advance. * 1. Docker is another example, that API changes almost every month. The driver we have is unstable and doesn't support API versioning correctly. * Please consider these options, I would like to hear from users how you are currently using Apache Libcloud and how you are using it NB: I have mentioned Azure, AWS and GCP a lot in this thread, mainly because they represent +80% of the cloud market [3]. This is a community developed project, has no affiliation to those vendors and these are my opinions, not those of the project. [1] https://www.amazonaws.cn/en/products/ [2] http://www.computerweekly.com/news/450280991/IDC-research-predicts-gradual-decline-in-on-premise-hardware-spend-as-cloud-adoption-rises [3] https://www.crn.com.au/news/microsoft-ate-into-aws-market-share-in-q4-481240 [4] https://docs.google.com/spreadsheets/d/17KEs8Lr_bCQ1XI7QzqmNVe279d7-vPIYPer2VFtSo_Q/edit?usp=sharing
