Please share with the community what you think needs improvement with Amazon EMR.
What are its weaknesses? What would you like to see changed in a future version?
Amazon EMR is continuously improving, but maybe something like CI/CD out-of-the-box or integration with Prometheus Grafana.
The most complicated thing is configuring to the cluster and to ensure it's running correctly. You need to configure at least three Amazon policies to get authorization for all the instances. And if you're new on the system it's really complicated. It's something that could be simplified for users. For additional features, I'd like to see a better MLOps platform but it's possible that it's already in production.
The dashboard management could be better. Right now, it's lacking a bit. I'd like more of a remote connection between my computer and the solution. We have multi-factor authentication, and at one point it was an issue due to the fact that I lost my phone. It stopped me from accessing the system. We have to replicate all the infrastructure and we need to ensure that we have the scalability and to do so in production. We are hoping that Amazon will allow us to scale easily. However, we have not attempted to scale just yet.
How do you or your organization use this solution?
Please share with us so that your peers can learn from your experiences.