Engagement Model for Machine Learning Projects

This is the result of improperly selected Engagement Model 

MindCraft Case : A customer turned to us for a consultation. He had been working with a Data Science developer for 6 months, with no result. However, it was crucial for him to present the work in progress and show his investors that the product is real. After we analyzed his problem, we found two drawbacks in his initial plan. First, he described his requirements for the Data Science solutions as if it was an ordinary software product. Second, he hired a junior specialist to deliver it. As a result, he spent two times as much money as he had planned before and wasted 6 months. Our team advised him to choose a more suited engagement model and in 2 days the client got a working PoC. To say that he and his investors were satisfied would be to say nothing. Here you can find out more details about this case.  

What does the Engagement Model mean and why is it so important?

Engagement model determines the basis for cooperation between a client and a vendor. For example, in software development, it defines the most mission-critical aspects: how the project will evolve (in a linear or flexible way), how the team will be assembled (hand-picked by the client to work exclusively on the project or assigned by the vendor), how the payments will be made (in a fixed bid, monthly, weekly, etc.). From our experience, a wisely selected engagement model can increase the chances of project success by 50%. If you choose the wrong one, it can create obstacles that the project will never be able to overcome and recover from.

The main functions of Engagement models are the following:

• Defining strict roles and responsibilities on the project
• Evaluating the risks and assigning people in charge of them
• Establishing rules and setting the boundaries
• Describing the general plan and strategy of client-vendor cooperation

What Makes Data Science Projects Different

In the case with a Data Science-based project, selecting the right engagement model is not just important, it’s vital. So here’s how projects focused on Machine Learning, Artificial Intelligence and similar domains are different from the rest of software development projects:

Research.

Being a rather new science, Data Science always involves plenty of research. The project is impossible without research and experiments, as this is what’s being done before getting to the actual development. Even the tasks that have already been solved by someone still require new approaches, as every data is unique and one of its kind.

More than software engineering.

Data Scientists are not software engineers. Machine learning Engineer is a software programmer with the skills of a mathematician. Data Analysts won’t survive without the knowledge of business analysis and mathematics. Data Engineer has to know how to work with data and also be aware of all aspects of the software development project, where the data solution will be integrated. The perfect combination is a Data Scientist, who knows every detail about your software, understands your business needs and works with your organization from within, not on the surface.

Keeping up with the industry pace.

As in any other science, advances in Data Science are described in numerous articles and scientific works coming out every day. Professional Data Engineers spend at least 30% of their time on self-development. They get familiar with innovative approaches to be able to integrate those in the clients’ projects. At Mindcraft we invest time, effort and money to make sure our team undergoes rigorous training and demonstrates both a deep knowledge of technology and a thorough understanding of business processes. That is why at MindCraft we always consult our clients on the advantages and risks of every engagement model. And we do it in advance. Discussing the details upfront helps partners to see the project with each other’s eyes and better understand each other.

And now we will try to give you the gist of the most common engagement models that are used in Machine Learning solutions development. We will also give you both the customer’s and the vendor’s viewpoint.

Fixed Price Engagement model for Machine Learning project

As you can see from the word “fixed”, this model means a fixed scope of tasks, fixed project requirements, fixed budget. In short, you pay the fixed price for the fixed scope and predetermined project flow. If you choose this model, be ready to work out your project in detail: describing every stage and action on the project, every task and its requirements and acceptance metrics. From the moment you start working on your project based on this model, there will be no chance to change anything. The changes will likely cause additional expenses and postpone the release date, which can mean significant damage to your business.

If you’re wondering why then some people select this option, then we can single out one possible reason – an opportunity to control your budget, understand and predict all expenses.

How does it work in Data Science? We always say that there’s a huge difference between software development and Data Science. All projects involving or somehow connected to Data Science fit in the Research & Development category. Research here is immensely important because the entire science stems from research. So for Machine Learning projects an ability to research freely is vital.

Let us summarize the advantages of the Fixed Price model:

• Budget. You know the project budget in advance and you don’t risk facing an increase in expenses unless you change the project requirements.
• Timeframes. You have a strictly determined date marked on the calendar saying when the project will be ready.
• Results. From the very beginning, you know exactly what the expected results and their success criteria are.

And here we’ll list the main concerns that may arise while working with this model:

• Lack of flexibility. There is no way you can change project requirements and timelines. If the project starts, you can’t do anything. Any change will lead to negative consequences: increasing the development time and budget.
• Piles of documentation. You need to carefully develop and describe your plan and requirements list to be successful. If you need to implement any changes, it also should be reflected in the documentation and confirmed by the other party, which can take a few days or even weeks.
• Limited project size and complexity. Usually, large-scale projects don’t work according to the Fixed Price model. The bigger the project is, the more you’ll have to think over and describe in piles of documentation. We’ve seen projects, where the process of describing specifications and creating project documentation was dragging for 2 years. In a rapidly changing world of innovation, this could mean that somebody’s already taken your idea and brought it into life. We even had a case, when we showed the client a working analog, offering to develop a custom version of it. Why did this happen? Because the selected engagement model was wrong. Keep reading to find out how to select the right one.

How should you handle big projects using the fixed price model?

In this case, we recommend dividing the whole big work into smaller chunks. Start with smaller expectations, set more reachable goals and increase your results by days and weeks.
Here’s an example of a typical Machine Learning project. The client wants a product which can recognize street signs from the photo. The recognition accuracy must be at least 95%. The dataset is a list of photos made manually or taken from Google Maps, etc. The algorithm should also be able to understand what kind of sign it is, what the road markings are and pass this information into a special device worn by the user.

This is a rather complicated task since numerous nuances have to be taken into account: what country the sign belongs to, how much time the processing can take. The customer wants it to work in a live mode, in real-time. Which language do these signs use: German, English or Spanish? Factors like that have a huge impact on the development time and price of the project.

And now let us try to divide the project into phases:

1. Is it a sign or not?
2. What kind of sign is this?
3. How accurate is this statement?
4. The decision-making speed. 

These are the kind of steps we use to help our customers control the development process and the expenses.

In reality, it looks like this.
For the first week, we work on the model which will help us determine whether it’s a sign. In this task, the goal is very clear and controllable, which helps to easily draft the budget as well. Then we recommend raising the stakes and starting to work with the quality and the types of signs. Here the client can choose a mixed model since the process of perfecting the accuracy is quite long and tedious. Sometimes we would need a few weeks to reach 90% of accuracy, and 91% might take months. So on this stage, we need to be perfectly aware of your business needs.

It is important for the task to last between 1 to 5 days, not more than that. This way, the client will be facing minimal financial risks and will be able to move on if the result is what they expected.

We’ll be happy to explain this approach to you and divide your project into chunks. Because we believe it is the best way for big projects to be able to influence the development with minimum risks while staying on the Fixed Price model.

Time & Material Engagement model for Machine Learning project 

This model means that you pay only for the number of hours spent on your project. It works best with innovative tasks, which rely on extensive research. This model allows to get results fully proportional to the money invested in the project. On any stage of cooperation, you can control your resources, increase the number of development hours to quickly develop a product or decrease if you need to reevaluate your budget.

To leverage this model successfully and efficiently, you need to understand, that it demands your full involvement in the development process. You always have to be in the loop, know how much time and money you spend and which results you expect. And to do this, you need to be the one managing your development project in the live mode. Luckily for you, modern Agile methodologies will help you get maximum value for money.

Agile methodologies in Data Science projects 

We often work with this particular model. It has proven itself to be the optimal one for startups looking for investors. It allows to rapidly develop a Proof of Concept, which you can bring your investors to attract finances.

Perhaps one of the biggest advantages of this model lies in the fact that you can follow the trends. At any moment of time, you can try new approaches and remove outdated practices. It will help you stay up-to-date, while also making sure you invest in the right things.
In addition, you can easily combine tasks lasting from 1 to 5 days (not more), with minimal financial risks, and you can move on only when satisfied with the result.

Our clients who chose Time and Material often admit that they enjoy the full control over the project flow and constantly having their finger on the pulse of what’s going on.

Here you have to know exactly when you’re planning to reveal your product to the world or start using it.

Because the main danger here is that there’s no limit to perfection. Often the release date is postponed due to the project management not being done right, and a gap between “ready to release” and “would be nice to have”. This is something we always warn our clients about. Our data scientists have project management and product management experience, which allows to look at the project at a wider angle and eliminate risks like that.

Dedicated Team for Machine Learning project 

This model is fairly simple. You hire engineers who are exclusively engaged on your project, either as a standalone team or as an extension of your in-house team and get paid on a monthly basis. It’s a top choice for people who have in-house data scientists and either search for some specific expertise that they’re missing or need help coping with big amounts of work. That is why they are ready to outsource some of their tasks. Outsourcing is cheaper than hiring a new team, for much less time you get a ready-made seasoned team of specialists who are already comfortable working together.

Here are a few cases when this model works best:

• If you need more Data Science talent, but cannot afford hiring in-house and would like to maintain the same high quality for the lower price.
• If you’re only planning to build your Data Science department or even need this expertise temporarily, to accomplish certain tasks.

An in-house Data Science department is quite a costly initiative. On the other hand, there’s no guarantee that you’ll always be in need of data research and that your budget will accommodate these expenses. That’s why we recommend having an outsourced team of data specialists at your fingertips. You can engage with them on an on-demand basis, saving tens of thousands of dollars and paying only for the work you get.

Conclusion

We showed you the three most popular engagement models for Machine Learning projects. Every model has its pros and cons, dos and doesn’t, which you need to be aware of before starting to work with any of them. Years of industry experience have taught us about many hidden threats and shortcomings. We always warn our customers about them, to make the result is predictable and within their reach.

If you have any questions or concerns about your project, reach us at team@mind.local or by writing to Andy Bosyi, Mykola Kozlenko or Nazar Savchenko on LinkedIn. Before getting to the actual work we always provide free consultations to keep our clients aware of every aspect of our future projects together. Many of our customers mention it (while referring us to their friends and colleagues) as the reason they entrusted their projects to us in the first place. We always think about the client’s business needs and do our best to deliver truly custom-tailored solutions

Also read : AI & Machine Learning solution for Business: Getting Started

Author:
Nazar Savchenko, OD at MindCraft
Information Technology & Data Science 

you might also like…