HPE Code Editor Experience

Machine Learning Data Management (MLDM) is software that makes feeding data to AI models easier and provides the history of changes so that users can experiment safely. Our business’s goal was to transition from being a tool used to experiment and create proof of concept AIs to being a tool that can be used throughout the entire AI development lifecycle. This case study looks at two interconnected projects:

Giving users the ability to apply default settings if they have the right role based access controls (RBAC).
The code editing experience that allows user to apply these default settings, and to write code for preparing data for AI use.

The Problem

Timeframe

Role

Solo designer

8 weeks (2024)

Conflicts and Outcomes

New MVP backend technology, and the missed opportunities.

The primary piece of new technology allowed admin level users to create defaults for critical infrastructure settings (an example of this would be telling your AI to compute only using Intel cpus instead of AMD). The intention was to alllow the admin to set the default, but to allow users to overwrite those defaults if they had the proper permission. While this tiered access control is not new, because of the MVP nature of the feature I had limited options because of a lack of APIs. Options like a conventional settings page couldn’t be supported because of this lack of APIs.

Outcome

The only feasible option to roll out this new feature was to combine it with a code editor project. These same settings and defaults are already created in the code that AI developers write. But, there were absolutely missed opportunties to provide a better experience for both writing code and creating these setting defaults because of the lack of APIs and the lack of developer capacity for this project.

Working within the limitations

Despite the sub optimal circumstances, using a code editor as a vehicle was able to meet our goal of providing a RBAC restricted code editing experience that allowed admins to set defaults and users to overwrite those defaults when necessary. The major obstacle to success was a lack of budget for user research. To help mitigate bad outcomes, I leaned heavily on feedback from the support team; however, that came with the risk of designing a solution for the support team and not the users of our product.

Outcome

Although there were UX issues with the final product, the end result was still successful in making writing code for AI easier. Design patterns from GitHub and VS Code provided the backbone of the design solution, especially the use of IntelliSense which we were able to hook up to our code editor experience to provide autocomplete suggestions.

New MVP backend technology, and the missed opportunities.

Summary: Both our RBAC system and admin settings experience were brand new features built in the backend and still had issues with its implementation. Its newness meant that there were gaps in the APIs which didn’t allow our front end to easily access information in the backend. This heavily restricted the experience that I could design.

A major challenge with these projects was that the back end for these new system were being designed and built while I was designing the UX and UI. Several assumptions about the final product needed to be made.The backend and frontend teams were asked to release these features simultaneously due to customer and user pressure.

I took steps to mitigate as many of the risks as possible which included:

Being present for the backend planning and design meetings.
Involving the backend and support teams during design reviews, and co-creating with them.
Using async reviews in slack to keep consistent communication about the direction of the UX/UI designs, and soliciting on the fly feedback.

This is a simplified summary of the roles and permissions with regards to default settings. The important interaction to know is that defaults can be overwritten by users.

Success Criteria:

Users need to be able to write code for their pipeline specifications.
Admins need to be able to set defaults at the cluster and project level.
Non-admin users need to be prevented from setting defaults.
Users need to know what defaults are in effect in their code, and be notified when they overwrite defaults.
Admins need to be able to overwrite code in bulk.

The combination of success criteria lead to projects that were complex in nature due to how interdependent a user’s actions could be.

A user with the proper permission can overwrite the defaults, and an admin can decide to overwrite some of that user’s code when appropriate. An admin can even overwrite and rerun the data processing. Because this is a destructive action I needed to design a way to disclose what would be impacted. I settled on a table that shows all the impacted pipeline steps and projects and users could click through to view the project or pipeline in a new tab.

The missed opportunities

In a perfect world, using the common pattern for a settings page would have been an excellent solution to the settings experience we wanted to create. However; with the lack of API support, this wasn’t a feasible option. The default settings that admins create can be breaking changes. It is worth it to describe the various settings and how they impact their AI model development and maintenance. (Above is the settings page for Linear).

Adding a code editor experience was a stopgap solution. We we wanted it in our product, however; I knew it was not the optimal solution. The ideal solution wouldn’t take the user out of their preferred code editing experience. Displaying how defaults are being overwritten should have been a plug-in experience, but that meant that we would have to engage the plug-in team who were full on their roadmap for the next 3 fiscal quarters. And again, the lack of API support prevented this option. (Above is VS code which is what most of our users use for code editing).

Working within the limitations

Summary: Given the limitations, I focused on what was possible, using code editing software as the primary source of inspiration. I focused on experiences that developers were already familiar with to drive these projects to completion.

To recap the previous section:

The admin settings experience needed to happen in a code editor experience because of the lack of API support.
The lack of API support also meant that the better experience for code editing (plug-ins for a developer’s preferred code editor) was also not a feasible option, and the plug-in would not be available for another 2 fiscal quarters.

The iterative approach:

The initial concepts were put together with screenshots of ideal features from other products (VS code, settings pages, etc.). The team scaled back based on what was feasible.
Supporting features were designed to support the key requirements. These supporting features were penciled in as tentative because the team was still waiting on the final product from the backend team.
I facilitated a concept sign off which is a meeting I run where the team agrees on the information architecture and the overall experience with the understanding that detailed design work will be contingent on feasibility, and edge case work will happen after the concept sign off.
The backend and frontend team continued to feed me information about what was feasible and I adjusted design accordingly.
I performed quality assurance on the internal build of the feature as they became available.
Shipped!

The final product of these two projects included a code editor experience that allowed users to write their pipeline specs, cluster defaults, and project defaults. It also included Intellisense for on the fly correction of JSON, error management, and other quality of life features you would expect from a baseline code editor experience.

Examples of the supporting features: icons that denote where a user has overwritten a default, read only tabs for the created defaults, a toggle between JSON and YAML, a side-by-side view, and many other features.