Touchless Chef

50.001 1D Project

A POC Recipe Android Application using Google Mediapipe Hands Tracking model for smart touchless scrolling.

Deliverables:

Download and install our .apk HERE to try it out

Slides

Poster

Video

A. Acknowledgement:

Tran Nguyen Bao Long

@TNBL

: Project Lead
- define software architecture
- documentation
Li Xueqing

@cnmnrll

: Project Design
- set basic Recipe, Ingredients and Instructions models model
- documentation
Melodie Chew En Qi

@melmelchew

: Marketing + Frontend
- RecyclerView and NavigationView
- adapter for RecyclerView
Melvin Lim Boon Han

@melvinlimbh

: Marketing + Frontend
- Fragments for different cuisines home
- Fragments for creating recipes RecipeCreate
- all resources and layouts res
Han Jin

@dlmszhhh

: Backend Acitivity
- basic recipe activities acitivity
Wang Zhouran

@wzrwzr23

: Backend Database
- setup SQLite database and corresponding Data Access Objects dao and adapter
Liang Junyi

@LiangJunyi-010

Backend HandTracking
- implement HandTracking using Google Mediapipe Hands model for touchless scrolling in ViewRecipeActivity

B. Description

Overview of Design Pattern used:

Adapter Design Pattern (for SQLite database and RecyclerView)

Singleton Design Pattern (for SQLite database)

Observer Design Pattern (for CreateRecipeActivity)

Factory Design Pattern (for Cuisine Fragment)

Template Method Design Pattern (for Create Recipe Fragment)

0. Model

UtilizeParcelableinterface:
- speed up data transfer using Java Serializable
- easily pass data as Bundle object through Activity
Recipe :
- id: primary key for SQLite
- name: name of recipe
- description: short description of recipe
- category: category/cuisine (Vietnamese/Chinese)
- mealType: type of meals (Breakfast, Dinner,…)
- time: estimated to prepare the dish
- imagePath: local URI path to Recipe image
- ingredients: list of ingredients
- instructions: list of instructions
Instruction :
- id: primary key for SQLite
- recipeID: foreign key
- content: instruction body
Ingredient :
- id: primary key for SQLite
- recipeID: foreign key
- name: ingredient name

1. Database

We use local database SQLite and utilize abstract class SQLiteOpenHelper to create and manage our database in SQLiteDbCRUD

Adapter inDatabaseAdapter
- using Adapter Design Pattern as main API for Recipe CRUD operations
- using Singleton Design Pattern to create a single instance of our database to be shared within all activities
Data Access Objects (DAO):
- provide an API for our app to set/get data from SQLite database
- map SQLite Config with corresponding model attributes
- Config is a nested class improving encapsulation since it is only used within dao package for CRUD operations

2. Activity

Both MainActivity and ViewRecipeActivity inherits a Toolbar (explain in section B5a) from abstract class MenuActivity
MainActivity
- implements interface BaseFragment.FragmentListener that define blueprint for methods to:
  - display recipe: onShowRecipe which navigates to ViewRecipeActivity (an Explicit Intent)
  - delete recipe: onDeleteRecipe
- consists of 3 main views:
  - ImageView show background image for selected Cuisines
  - DrawerLayout for NavigationView (explain in section B5b)
  - NestedScrollView for displaying of recipes for different Cuisine Fragments (explain in section B4a)
ViewRecipeActivity:
- display chosen recipe from the data passed as parcel from MainActivity
- where HandsTracking feature (explain in section B3) is located
CreateRecipeActivity
- implements 3 Recipe Create Fragment listener, each corresponding to 1 step in creating the recipe (explain in section B4b)
- using Observer Design Pattern, changing Fragment when reaching subsequent step in creating Recipe

3. Hand Tracking

This feature is implemented in ViewRecipeActivity
- 2 main Buttons Start Tracking and Stop Tracking to enable/disable the feature, which open up a glSurfaceView in the RelativeLayout at the bottom of the page
- glSurfaceView is an instance of SolutionGlSurfaceView<HandsResult> provided by Mediapipe Library, similar to OpenGL that allows writing words and drawing on webcam output.
- we detect whether the index finger is pointing up or down and from there, post a Runnable to our ScrollView (that is displaying the Ingredients and Instructions) to perform a smooth scroll correspoding to the gesture.

4. Fragment

a. Cuisine Fragment under `fragment/home`

Both ChineseFragment and VietnameseFragment extends from abstract class BaseFragment
- using Factory Design Pattern (BaseFragment is the factory, ChineseFragment and VietnameseFragment are the products) allow us to easily scale to multiple cuisines in the future
- the only abstract method is getFragmentLayout that each product should define to generate its own correct fragment
- both using RecyclerView to display its recipes (explain in section B5c)

b. Recipe Create Fragment under `fragment/recipe`

3 fragments correspond to 3 steps in creating a new Recipe:
- RecipeCreateImageFragment: (1st step) to add image, description, choose meal type and time
- RecipeCreateIngredientFragment: (2nd step) to add ingredients
- RecipeCreateInstructionFragment: (3rd step) to add instructions

All 3 Create Recipe Fragments extends the abstract class NavigableFragment :
- blueprint for method onNext that define which fragment to navigate to after completing each creating new recipe step
- using Template Method Design Pattern with NavigableFragment as the template
RecipeCreateImageFragment can access the local storage to upload new image for recipe (an Implicit Intent)

5. View and Layout

Both MainActivity and ViewRecipeActivity needs to define its own Toolbar according to the blueprint defined in the abstract class MenuActivity
MainActivity Toolbar:
- includes NavigationView (explain in section B5b below)
- includes menu to add a “New recipe”
ViewRecipeActivityToolbar:
- support Edit and Delete recipe functionalities

make use of a switch case to switch to a different cuisine inside MainActivity
once a new cuisine is selected, the corresponding Fragment (ChineseFragment or VietnameseFragment) will be added into NestedScrollView to show recipe CardView

c. RecyclerView

We user RecyclerView at 2 places:
- to display recipes in the Cuisine Fragment (BaseFragment)
- to display the added ingredients and instructions when creating new recipe in Recipe Create Fragment
To implement RecyclerView we need to implement the corresponding adapter as shown in adapter/recipe (Adapter Design Pattern)
- the 3 main functions in each adapter to override are:
  - onCreateViewHolder that inflate a new view in the parent layout to allow for display of listed items
  - onBindViewHolder that binds the data to be displayed in as listed items
  - getItemCount that return the size of all the items to be displayed
- in the corresponding Fragment, before we set the above adapter, we also need to set a LinearLayoutManager for the RecyclerView to reside in
- code examples:

private RecyclerView instructionRecyclerView;
instructionRecyclerView.setHasFixedSize(true);
instructionRecyclerView.setLayoutManager(new LinearLayoutManager(getActivity()));
instructionRecyclerView.setAdapter(instructionAdapter);

C. Appendix

0. Reference:

We made use of a some very useful Youtube tutorials + Github repos to build a normal Cooking Recipe app before adding modifications and our hand tracking feature: