Over the past few years I have started to develop a personal framework for evaluating new technologies. I have had the good fortune of being able to work on many greenfield projects and many of these start from scratch so I have evaluated a lot of different technologies for many use cases. From this I have been able to really get a good feeling for what metrics help me decide if a technology will be a net positive for the project in the long term.
Clearly define the need
Above all else the needs must be clearly defined. This can be quite difficult on greenfield projects that shift rapidly as they are first being developed. For most projects I try to define one or two key metrics to help me shape the most important needs. Are we focused on performance? A great user experience? What about stability? You may not know exactly what your needs are today but you must choose to optimize for something. Once you have decided on what to optimize for you can then understand what needs rise to the top.
Maturity vs Popularity
I often see people reach for maturity as a significant driver - for many of my projects maturity is a bit more of a nuanced topic than I ever expected it to be. Maturity is not incredibly difficult to gauge by itself but you do have to understand whether maturity is a useful characteristic for your project. I have seen projects adopt mature technology but realize its not well integrated with other newer technologies they have chosen. Mature technologies often move slower than ones on the bleeding edge so choose accordingly. You have to understand your ability to support and potentially adapt the technology you adopt. It is not always the case that bleeding-edge technology is going to require more support than a mature technology.
Time to learn
When it comes time to adopt the technology you often are faced with introducing it to a significant number of team members. Understanding clearly how the technology fits into your stack and how challenging it is for people to get up to speed can not be understated. There is a saying that goes a 1 hour meeting for 5 people is not a 1 hour meeting, its a 5 hour meeting. The same thing applies to technology as well. Think about the number of team members that interface with the technology directly. If the factor is high and the interface is time consuming to learn you might need to build an adapter to shield people from that educational hurdle.
Blast Radius and backup plans
Sometimes we make a decision on a technology that does not pan out. Even worse that technology could make it into production and you have to move mountains to replace it. It can sometimes feel odd to evaluate how you would replace a technology before you even decide to adopt it but if something goes wrong the backup plan can come in handy. Having some sort of alternative that can be shimmed in if things are not going as planned can be a lifesaver - backups are always a good thing to have.