Proteus Motion

Status

Completed

Job Type

Full Time Internship

Technologies

Typescript, ReactJS, ThreeJS, BabylonJS

Role(s)

Front-End programming

During my internship at Proteus Motion, I worked on the 3D visuals for their latest device, the Proteus System. I have refactored their ThreeJS code into BabylonJS so that they can offload some processes into a separate thread using Babylon's Offscreen Canvas. This is desirable because it will help their biofeedback system run faster. By using the data they gave me, I was able to construct 3D spaces and objects that will help their users analyze their exercises. I also gained experience with converting React class components into functional components using hooks.

This part of the system allows the user to practice exercises, displays how much power they used, as well as show their movement for that exercise. I worked on adding the scene where you see the pink, orange and blue spheres and a purple tube connecting them. The tube represents the movement path. The orange and pink spheres are the start and goal points respectively.

The tube is the most important part of this section as it what connects the 2 spheres to each other. In order to make the path, I needed an array of points. Then I needed to use those points to make a curve which I would then pass into the function that makes the tube. The tube was tricky because I wasn't sure what kind of curve I needed to use. However, upon comparing the production code with Three and the available Babylon's curves, I decided that the CatmullRomSpline was the appropriate curve to use. The tube is comprised of multiple mini tubes, using 2 points at a time. Then I group them all into 1 object by parenting each mini tube to an empty parent object.

This part of the system helps the user visualize the amount of power they used during the exercise. They use a rainbow color scale where red represents high power while blue represents low power. The user can toggle the reps they did by clicking its button, hide/show all reps, and compare their current performance with prrevious dates using the buttons on the right hand side.

This is how I made the heat tubes. I was given a metrics objects which contained the positions for each individual sphere that made up the tube, as well as their recorded power. It's just a matter of looping through all the positions, creating a sphere at that position, and giving it a color based on the power. At the end I make the tube invisible, which will become visible when the user clicks the appropriate button.

Here is an aerial view of the scene, allowing you to see the shadows of the colorful heat tubes on the floor. The shadows help the user perceive the depth and positioning of the 3D objects.

The colors of each sphere that makes up the tube is determined using a number argument. Three and Babylon use different measurements for color (Three uses HSL while Babylon uses RGB). It was difficult, but I mangaed to find a formula that could convert HSL to RGB. The reason why it was difficult was because Three was using values from 0-255 while Babylon could only accept values from 0-1. Most formulas I found required you to be using values from 0-255.

After I did find a formula, I added that color onto a material and then added that material to an array. Once the sphere is created it will pull the appropriate material from that array.

Lighting the scenes was a challenge since I was not using a GUI and lights are invisible objects. To help with that, I wrote a function that give lights a physical form. It's really simple, I just make a sphere and place it where the light is. I also give it a color.

I used a spotlight to cast shadows since I can't use the ambient lighting (that type of lighting doesn't extend the IShadowLight interface). I instantiated a ShadowGenerator object and passed in the spotlight. Then I allowed the plane (the floor) to be able to receive shadows. I also made the shadows blurry to make them more realistic.

Hooks are great because they take the complexity out of class components and also reduce boilerplate code. So my next task was to convert the class components to use hooks instead. (Since I don't to expose how their data was structured, I've changed the names of some of the variables & functions but you will still understand the process I used for the conversion).

Here I used a combination of useState & useReducer hooks to separate each field of the state. That way I'll know right away which field is being updated. For simpler fields like strings, I would use the useState hook. For more complicated fields, like objects or arrays, I used the useReducer hook instead.

Here I have a useEffect hook that I used to mimic the componentDidMount & componentDidUpdate lifecyle methods. It is just getting some maps that are needed to render the 3D Babylon scene I created.

Here I have a useCallback hook, which was one of the functions in the class component that dealt with state. Any function that doesn't deal with state can just be a regular function declared outside of the component. If it does, it's better to have it as a useCallback hook so that when something in the state changes, the hook will take those changes into account.