Harnessing Augmented Reality for Increasing the Awareness of Food Waste Amongst Dutch Consumers

Dolf Honee; William Hurst; Antonius Johannus Luttikhold

doi:10.1007/s41133-022-00057-7

Harnessing Augmented Reality for Increasing the Awareness of Food Waste Amongst Dutch Consumers

Honee, Dolf; Hurst, William; Luttikhold, Antonius Johannus 2022-12-01 00:00:00 Food waste is a signiﬁcant challenge, and our societal behaviours play a role in the amount of food items discarded. Thus, an effective method to inform consumers about high wastage patterns may help reduce the amount thrown away. This research investigates how Augmented Reality can be harnessed to enlighten consumers and work towards addressing high food waste patterns. Yet research on this topic is still very much in its infancy. To pursue this solution, food behaviour data are employed to provide an insight into how much is wasted from 9 catering industry locations in the Netherlands. An Augmented Reality application is developed, where models of food are projected onto real-world environments to provide scale on waste over a 7-day period. A quantitative evaluation of higher-education attendees demonstrated the approach has potential to incentivise reduction in waste. Keywords Augmented reality Food waste Catering industry Consumer Introduction Understanding the volume of waste is a particular challenge for consumers, and digital technologies, such as Food waste is a signiﬁcant problem in the current economy. Augmented Reality (AR), could provide a supportive A considerable volume of food is thrown away or wasted. metric to increase awareness with the potential to inspire In the Netherlands, about 9.5% of all bought food is wasted consumers positively [4] by providing a visual aid to yearly; this equates to 34.3 kg of solid foods [1] and understand the amount of waste produced. AR is a suit- equivalent to 154 kg carbon dioxide (CO ) per person per able technology for this purpose, as it is becoming more year. Furthermore, this accounts for 8–10% of all the accessible and widespread, largely due to the capabilities greenhouse gasses emitted by the Netherlands [1]. In wider of smart phones. Because of this, in the current environ- Europe, about 186 Mt CO -eq is attributed to food waste ment, almost every consumer has an AR-capable device, [2]. This has resulted in food campaigns, such as the clean making the technology an ideal solution for visual com- dish, clean conscience! initiative in Portugal, to help munication. For instance, each smart phone is equipped increase awareness for food waste behaviours [3]. How- with a screen, camera and hardware, making the devices ever, in general, as conﬁrmed by UN Sustainable Devel- suitable for running modern-day AR applications. Fur- opment Goal SDG 12.3, more solutions are required to thermore, AR is already used regularly in education and address the pattern of waste. communicative applications. For example, according to [5], AR is a suitable method to deliver educational material due to its ability to deliver a learning experience that & William Hurst combines experiential, kinaesthetic and didactic learning. will.hurst@wur.nl Similarly, in a study by Nicholson-Cole et al. [6], visual Dolf Honee communication showed a positive impact on the perceived dolf.honee@wur.nl importance of the prominent subject of climate change. Antonius Johannus Luttikhold Thus, demonstrating that users are becoming familiar with thomas@wastewatchers.eu AR applications and that a visual element is an ideal metric Information Technology Group, Wageningen University and for communication. Research, Wageningen, The Netherlands Wastewatchers, Utrecht, The Netherlands 123 2 Page 2 of 12 Augmented Human Research (2022) 7:2 It may be challenging to grasp what it means to waste Search Strategy 8–10% of bought food, but a visual representation of this may have a clearer impact. The theory is that, through the The focus of the related work is on two digital libraries use of an AR experience, consumers may be more inclined IEEE Xplore and Science Direct. As AR is an emerging to both understand and work towards a reduction of food technology with continuous developments, only literature waste behaviours. In order to address this, the study from the last ﬁve years (2016–2021) is considered in this investigates suitable AR approaches (from related works) study. A survey of the literature landscape demonstrated for the creation a food waste application. An in-depth that a signiﬁcant amount of AR-based articles are confer- analysis of related work is conducted using a snowballing ence based. Therefore, the targeted sources include jour- technique, in which only open-access articles are consid- nals, conferences and workshops. For both libraries, a ered for repeatability of the investigation. Based on the focused general search query is used to identify papers ﬁndings, a prototype AR application for food waste visu- speciﬁcally addressing food waste and AR. The search alisation is designed and developed using the Aryzon AR catered for possible variations in the key words. This Software Development Kit (SDK). At the time of writing resulted in the following generalised search query, where this article, Aryzon is an emerging technology and yet to be 458 papers were identiﬁed (75 from Science Direct and 383 employed within the food waste visualisation domain. from IEEE Xplore). Therefore, the technology is not found within the related ðAugmented Reality OR Mixed Reality OR Immersive OR works discussion but is an ideal solution due to the inte- Virtual OR 3Dor XRÞ ANDðÞ Food waste OR Food loss gration with Unity game engine and that it caters for both ð1Þ screen-based and wearable AR deployment. Evaluation of the AR application involved participants attending a (1) is an abstracted example of the search string, as vari- higher-education institute, a similar approach to the work ations of Augmented and Mixed reality (e.g. AR, XR, MR) by Pinto et al. in [2] for the promotion of food waste were included in the full search. Initially, the search string reduction. However, access to this participant group was had a broad scope to ensure no potentially relevant research limited due to social distancing measures at the time of articles are missed. From the number of papers, the most testing. relevant ones were ﬁltered using the selection criteria The remainder of the article is as follows. ‘‘Related presented in Table 1. The selection criteria are applied Work’’ section provides a background discussion and lit- manually by reading the title and abstract of the studies. erature review of the current use of AR for food waste This reduced down the number of articles down to 15 (a applications (at the time of writing this, there are relatively full list is presented in Table 2). An immediate observation few articles on this topic). ‘‘Materials and Methods’’ sec- is that there are few articles available that combine both tion discusses the methodology adopted for the develop- AR and food waste, demonstrating that the work presented ment of a AR application, and ‘‘Results’’ section provides in the article is within a potential research niche. In the and overview of the implementation. The article ﬁndings following subsection, the ﬁndings from these 15 papers are are discussed in ‘‘Discussion’’ section, and conclusions are discussed. presented in ‘‘Conclusion’’ section. Findings Related Work Within the 15 papers, differing development platforms are employed for the construction of the AR applications. AR has a broad scope of applications, examples include However, 5 studies did not state which platform they used teaching chemistry in higher education [7], support for (mostly because the investigations within the 5 articles are spinal surgery [8] and assisting with building tunnels in applied to the general implementation of AR). The most construction [9], to name but a few. As discussed, the focus used platforms are self-created; an approach adopted by of this article is on AR in the food waste domain, which is a 4/15 articles, for example, the SensiMAR prototype by research concept that is very much in its infancy. There- Marto et al., [10] and the M5SAR by Rodrigues et al. [11]. fore, in order to design and develop an effective AR This customised approach is a suitable solution to work solution, ﬁrst an investigation is required into related works with, as the tools are speciﬁcally made for their intended in order to ascertain the appropriate features to employ and goal. As Marto et al. [10] discuss, the use of a custom-built obstacles to avoid during the design and implementation of AR technology is necessary because no comparable tech- the solution. In this section, an overview of the ﬁeld of nologies were available with the same features. However, a interest is provided. limitation of this method is that it has signiﬁcant 123 Augmented Human Research (2022) 7:2 Page 3 of 12 2 Table 1 Study selection criteria No Criterion SC1 Papers without full text available SC2 Papers not written in English SC3 Duplicate publication from multiple sources SC4 Papers do not relate to AR SC5 Papers are not directly related to this study (i.e., not applicable to an AR app for food waste) The range of the categories indicates how ﬂexible Table 2 Study selection criteria AR is for deployment in varied settings. Table 3 dis- Fuchs et al. [12] 2020 Li et al. [13] 2020 plays the full ﬁndings and identiﬁed obstacles from the Fujimoto [14] 2018 Marto et al. [10] 2020 selected studies. Crucially, of the 15 papers analysed, Gong et al. [15] 2021 Qiao et al. [16] 2019 none is in the domain of food waste speciﬁcally, but Guillen et al. [17] 2021 Rodrigues et al. [11] 2019 there were some synergies; for example, Fujimoto et al. Hoppenstedt et al. [18] 2019 Shea et al. [19] 2017 [14] discuss the perceived deliciousness of food and Iba´n˜ez et al. [20] 2016 Song et al., [21] 2020 how this can be increased by chroma and colour Kim et al. [22] 2021 Zhou et al. [9] 2020 modiﬁcations. This would be in line with the ﬁndings in Koutitas et al. [23] 2020 Marto et al. [10] where a demonstration is provided that additional sensory stimuli may impact the experience of a user within AR. In terms of engagement, Hoppenstedt et al. [18] development costs due to the high levels of customisation. discuss that immersive analytics by means of AR are a 3/15 studies used Web AR as their main technology for suitable replacement for 2D representations of data and creating an application. Web AR is a ﬂexible approach, as may be even better for retention. Furthermore, Fuchs no app has to be installed and is compatible with almost et al. [12] highlight how the visual element of AR has any device that is AR-capable. A disadvantage of this an impact when users make a choice about which food method is that, at the time of this investigation, Web AR they will buy. For the design of any future food waste does not have a broad range features compared to other applications, this may suggest altering the 3D models custom-built AR platforms, such as AR Core. Other plat- with chroma and partial-colour modiﬁcations to help forms that are used in the articles include Unity AR, make the models look more appetising and incentivise Vuforia, and Magic Leap, as displayed in Fig. 1a. The users to waste less of these foods. Therefore, in the application domains of the 15 papers can be grouped into 8 application detailed in ‘‘Materials and Methods’’ section, categories, cultural, education, entertainment, food and the data will be presented through a combination of 3D diet, manufacturing, medical, sustainability and technol- models and 2D textual elements that further elaborate ogy, as in Fig. 1b. on the 3D models. In this way, the user is engaged by Fig. 1 a AR methods used b Research Domains 123 2 Page 4 of 12 Augmented Human Research (2022) 7:2 Table 3 Related work summary Study ID Findings Obstacles AR Research Features Methods Domain Fuchs et al Mixed reality (XR) can support healthier XR interventions are a promising Unity AR Food and Diet Wearable food choices ﬁeld of research foundation device ? App Fujimoto Digital representation of food may relate Limited number of food types Web AR Food and Diet No wearable et al to its perceived deliciousness considered in the investigation device, no app Gong et al Virtual reality (VR) may have a Application depends heavily on Vuforia and Manufacturing Wearable suitable user acceptance with the user acceptance of the VR Unity device ? App appropriate framework hardware Guillen et al Gamiﬁcation may offer user engagement There is a ﬁne line between Not Sustainability No wearable beneﬁts within sustainability subjects gamiﬁcation and game-based speciﬁed device ? App activities – and the long-term impact is inconclusive Hoppenstedt AR analytics may be a A wearable headset may be a Not Medical Wearable et al suitable replacement for 2D data– A possible stressor speciﬁed device ? App wearable headset may improve user engagement Iba´n˜ez et al AR can support with learning and Further study is necessary Own Education Wearable information retention in education platform device ? App Kim et al User perception in AR may depend on Perceived immersion may be Magic leap Technology Wearable interaction between virtual and physical disrupted when virtual objects device ? App overlap with physical objects Koutitas et al Reduce both computational resources and For best performance, caching Not Technology Wearable the induced network trafﬁc, by caching algorithm requires detailed speciﬁed device ? App images information about the user and object locations Li et al Web-based mobile AR and VR loading The number of animation tracks Web AR Technology No wearable times and experience can be improved and the size of the models device—No through on demand loading of 3D affect the optimisation of this app models method Marto et al Combination of sensory stimuli (audio, Impact of sensory stimuli on Own Cultural Wearable visual and smell) may improve overall experience changes platform device ? App enjoyment and knowledge retention in widely combined with other AR stimuli Qiao et al Web AR has a promising future for Depends highly on internet Web AR Technology No wearable mobile AR speeds -Web based AR is still device—No in its infancy stage app Rodrigues Social inﬂuence, effort expectancy, and Research only limited to one Own Cultural Wearable et al facilitating conditions are important in type of AR technology platform device ? App user acceptation in AR technology Shea et al Smartphones and their pervasive sensing AR apps with heavy pervasive Own Entertainment No wearable possibilities can result in successful AR sensing consumes signiﬁcant platform device ? App applications battery power Song & Qiu AR teaching methods may improve the Only investigated in political Not Education No wearable learning progress of contemporary and ideological education – speciﬁed device ? App college students Not a replacement for conventional education Zhou et al Different display technologies may affect In terms of more complex Not Technology Wearable users’ visual comfort, interaction learning activities and long- speciﬁed device ? App experience, learning experience, and term learning further study outcome in hands-on learning needs to be done the stereoscopic images, where speciﬁc information (that Regarding the hardware, one item of note is that a would be difﬁcult to represent in a 3D model) is pro- wearable device may be a possible stressor for users and vided through text overlay. this should be factored in when developing an AR 123 Augmented Human Research (2022) 7:2 Page 5 of 12 2 application for food waste awareness. The evaluation is present or the user taps to a location on the screen, the process in this article takes note of this, by offering a application will gather the required data from an internal testing process that makes use of both a screen and headset, database and start to project the 3D visualizations in AR. allowing the user to choose. This approach also aligns with When the user is ﬁnished, they have the option to close the 9/15 papers, which investigated an application and a app or start a new QR code scan. wearable headset combination, whereas 3/15 made use of only an application and no wearable headset, and another Data 3/15 made use of no installed application and no headset. Yet, according to Fuchs et al. (2020), headsets help to To ensure that the application and visualisation produced achieve high presence and immersions for the users. are based on a real-world scenario, a dataset containing Moreover, a wearable headset may improve user engage- 40,000 entrees from nine different locations in the ment [18]. An installed application has the advantage that it Netherlands from 01 August 2020 until 29 March can have 3D models pre-installed and allow for more 2021–2020 is employed (due to the signed agreement with complex features (an example of this can be the case of the data use, the locations cannot be documented in the Poke´mon Go! [19]). article to maintain privacy). At the time of writing this article, no AR technologies Within the data, the location, date, price and wasted within the articles evaluated have been developed speciﬁ- item by category are detailed. Artefacts (measurement cally for food waste. Based on other approaches in different errors) were ﬁltered from the raw data through consultation categories, the ﬁndings indicate that the most suitable op- with the data provider. An example of the data can be seen tions for the design of an AR app for food waste would be below in Table 4 (where labels A, B and C are employed to to create an app using a custom-built SDK (e.g. Aryzon). maintain the anonymity of the locations). This would cater for ﬂexibility, both for the design and When comparing waste measurements within the entire implementation process, as much of the framework and dataset, as in Fig. 3, bread and snacks are visibly the code libraries are already laid out. highest amongst the categories of left-over waste in all locations. However, bread and snacks are also the most sold categories. Materials and Methods Therefore, the percentage wasted compared to the units (sales) prepared by the cafeterias is factored in. The rela- The design for an AR application for visualising food tive waste is calculated by dividing the total units (sales) waste is put forward in this section. The approach is based wasted per category by the total units prepared per cate- on the ﬁndings from the literature review in ‘‘Related gory. The formula is used to calculate this, as follows in Work’’ section. Speciﬁcally, the design framework and (2): data used for the application are presented. The section is Total units of waste Lative waste per prepared unit ¼ 100% concluded with an overview of the hardware and 3D con- Total units prepared struction process. ð2Þ It becomes clear that bread and snacks are not the largest Design producers of waste compared to their amounts produced. This is evident in Fig. 4a and b, where, with almost 30%, A business process model and notation diagram is created grains are the highest waste producer relative to units below in Fig. 2 to represent how the app will function. The prepared. Aryzon SDK and the Unity platform are selected for the development of the application. Unity allows for custom For example, relatively fewer grains are purchased by the cafes and, thus, the total waste is not as signiﬁcant as, creation within the application, whereas the Aryzon SDK for example, bread or soup. Waste data are also split into supports the deployment process. In Fig. 2, when the user categories per days of the week. Tuesdays, Wednesdays, opens the application, they land on the home screen of the Thursdays and Fridays are very similar in their waste application. From here, users have the option to scan a QR distribution. code (if necessary) and open the camera or close the In Fig. 5, boxplots are used to compare waste distribu- application. A QR code is the most suitable approach for tion between days of the week, with food category on the interaction in a dynamic environment, such as a kitchen. x-axis and units wasted on the y-axis. A difference is Other approaches include using marker-less, which is the evident between the ﬁrst day of the week, last day of the approach adopted for the trial in this article. This is because week and the weekend. On Friday, the amount of meal the Aryzon technology is able to track ﬂat surfaces with a suitable accuracy for the evaluation. When a valid QR code salads wasted is often higher. Monday sees more waste in 123 2 Page 6 of 12 Augmented Human Research (2022) 7:2 Fig. 2 Framework Diagram for the AR application Table 4 Example data Date Location ID Product ID Amount Produced Amount Wasted Price Day Category (Euros) 29–03-2021 A 278 1 0 1,72 1 Snacks 29–03-2021 A 531 2 1 1,15 1 Snacks 26–03-2021 A 927 6 0 5,31 5 Hot meal 26–03-2021 B 277 38 5 2,03 5 Snacks 26–03-2021 C 2349 8 7 0,91 5 Dairy Visualisation In order to represent this information within an AR appli- cation, the visualisation will be based on 3D models ren- dered in the Unity engine, where a prototype is developed with the Aryzon AR Studio SDK. The AR tracking from Aryzon in the application is able to recognise ﬂat surfaces, without any major loading interruptions or errors. To minimise processing power demand, instead of rendering each amount separately, waste amounts will be categorized in 3D models that represent different amounts. For the Bread category for example, this means there will be a model for 1 unit of waste (e.g. one loaf of bread, or one snack item etc.), a model for 15 units of waste and a model for 25 units of waste. Conceptual examples are given for Fig. 3 Food waste by category for 9 locations bread, drinks and fruits and vegetables in Table 5, and displayed in Unity in Fig. 6. Whilst drinks are include for the soup category. On Sunday, it is very apparent that the the conceptualisation, only food is involved in the ﬁnal category Ingredient Level is wasted more compared to application prototype. other days of the week. A clear visualisation is the key feature of the applica- tion. A user should be able to identify what the 123 Augmented Human Research (2022) 7:2 Page 7 of 12 2 Fig. 4 a Food waste per category. August 2020 – March 2021 b Relative waste per prepared unit per category. August 2020 – March 2021 visualisation represents and have an idea how much it is. It is, therefore, signiﬁcant to make sure the 3D models are projected in realistic dimensions. Along with the models, additional data (in the form of text) are provided to support the visualisations. To recreate an appropriate level of realism, all the models have realistic textures that resemble the real-world equivalent. Hardware For the application to work, an AR compatible phone is employed. This means for IOS devices running IOS 11 or later and for Android devices running Android 7.0 or later. Furthermore, the application is developed for use both with and without the Aryzon headset as displayed in Fig. 7. Fig. 5 Boxplot of waste by category for 9 locations Table 5 Waste units Amount (units) 3D Model (Bread) 3D Model (Drinks) 3D Model (Fruits & Vegetables) represented by 3D models 123 2 Page 8 of 12 Augmented Human Research (2022) 7:2 Fig. 6 First trials with 3D models in unity Fig. 7 Aryzon’s cardboard headset a Side view b Front view into visor The Aryzon headset is constructed from cardboard, 19 quantitative involving use of the Likert scale and 4 which houses a mobile phone below the eye-level. Pro- qualitative where text could be inserted (an overview of the jections are created using an optical see-through approach, questionnaire is provided in Online Appendix A). where mirrors combine 3D model with views of the The participants of this test were characterized as fol- physical world. This means the real-world image is as lows: 19 testers, all of them experts with mobile phones natural as possible. The headset has a head strap for a and with more than three years of background experience hands-free experience. with smartphones and applications (aged between 18 and 24 years with, at least, a Bachelor’s Degree). Due to timeframe of the research and the current status of Covid- Results 19, the test group was limited in size and diversity. In the tests, users could install the Android Package (APK) on Testing is performed in the kitchen of the Leeuwenborch their Android smartphone or try the application out the IOS building from Wageningen University campus to evaluate smartphone provided by the authors. Users were introduced the real-world application (Fig. 8). 25 testers (students of to the topic and after the demonstration were explained higher education) were randomly approached on premise to what the visualizations represented. The visualisation try out the application. The process involved asking users provided for the evaluation only concerned food waste and to use the AR application, view the 3D objects and read the not drink. The inclusion of drink will be considered in text through means of viewing the application on a mobile future testing. phone screen. Out of these 25, 19 were available (access to a larger survey base was severely limited by the Covid-19 Features restrictions). 19 users also tried the AR application by using the headset for comparison with the screen-based When asked about features, participants got the choice to approach. Users were then asked to complete 23 questions, select every feature they favoured the most; no limit was 123 Augmented Human Research (2022) 7:2 Page 9 of 12 2 Fig. 8 AR application trial set to the number they are able to select (Fig. 9). 21% of observation was that the intuitiveness was chosen as the participants favoured the ease of use of the application the least as a favourite feature. When asked what users liked most. This may be a positive indication for the imple- the least, 33% said the textual information (Fig. 8). This mentation of the application within the catering business, may be a point to improve upon in further iterations of the as business professionals often may not want to spend time application. It is also notable that 25% of the survey par- learning a new application that keeps them from their ticipants chose ‘none’ as their most disliked feature. work. Second, with 18%, both the 3D models and the AR One user suggested to add CO emission information tracking were popular features. Overall, the most liked and another user suggested to add a menu. With regards to features were fairly widespread; where one apparent the immersion, all users felt immersed in the visualisation Fig. 9 Most liked/disliked features 123 2 Page 10 of 12 Augmented Human Research (2022) 7:2 but all mentioned that the immersion can be improved methods of data visualisation, all users preferred the 3D upon. A further suggestion was to increase the polygon visualisation of the application. Particularly for textual count on the 3D models (which would add more realism to reports and static images, this method for visualisation of the models) and another suggestion was to use more data may be preferred by about 70% of the testers. models in the same art style (e.g. to include drinks). The overall experience of the app was rated a 4/5 by the testers. The experience with the headset was the lowest with 3.4 Discussion out of 5 points. As discussed, food waste is a signiﬁcant issue in society, Food Waste and prominent particularly within the current catering industry; demonstrated by the data overview in ‘‘Materials When asked about this application incentivising for and Methods’’ section. Although 2D graphical representa- reducing food waste, 21% of the participants gave it a 5/5 tions may give an indication about how much food waste on a Likert scale and 37% gave the incentivization a 4/5. there is, a 3D representation would help with provide a About 16% rated this with a 3/5. 26% thought the appli- level of scope. AR has a broad scope of uses, but relatively cation was not very likely to have an impact. However, no few apps exist in the domain of food waste. In fact, to the users ﬁlled in the 1/5 option (Fig. 10). Notably, 60% of the best of our knowledge, at the time of writing this article no users were surprised by the information the app gave and applications speciﬁcally for AR with food waste exist. Yet, agreed that the application helped with understanding the AR applications in other domains have shown to help with level of waste. All participants agreed that the app clearly information retention and engagement, particularly in visualises which categories are wasted the most. education. From the literature review, it became clear that the Data Visualisation research in food waste and AR is still in its infancy. Most domains of the researched AR papers are within the tech- In terms of the usability to view the data, 42% would rather nology category, with 33%. 60% of papers made use of use the application without the headset. 26% would like to both a wearable device and an application. 27% of papers use the application with the headset and about 32% would made use of their own platform and 33% of the papers did prefer a hybrid approach (both with and without the not specify which AR platform they used. The rest of the headset), as displayed in Fig. 11. researched papers all used different platforms. One item of Regarding the question as to whether the visualization note is that a wearable device may be a possible stressor for provides a measure of how much food is wasted resulted in users. varied feedback. The most users (37%) gave this a 4/5 on A variety of platforms also exist for building an AR the Likert scale. The rest of the answers were divided application, each comes with their own advantages and between 2/5, 3/5 and 5/5. However, all participants indi- disadvantages. The most suitable option for this research cated that they were surprised by the amount wasted and seemed to be to create an app with a wearable headset and answered that the waste was more than they expected. The a custom-built platform. However, the use of an existing realism of the models was rated a 4/5, with some users SDK (e.g. Aryzon) caters for some ﬂexibility with indicating that the realism can be improved to improve the designing, as well as a relative quick development because overall immersion. Other suggestions for improvements in the framework is already laid out. the app were more diversiﬁcation between food categories Due to limited applications known about both subjects, and a clearer representation of the text. Compared to other this research is an opportunity to learn more about how AR can inﬂuence food waste attitudes or behaviours. The data are presented through a combination of 3D models and 2D textual elements that further elaborate on the 3D models. In this way, the user is engaged by the stereoscopic images and can get more speciﬁc information from text that would be difﬁcult to represent in a 3D model. During the devel- opment, it was chosen to use models that represent multiple units of food. A ﬁrst prototype was made and tested in various situations. Later on, models that represent all food waste from one week were implemented along with textual information overlay. The software used for developing the application was Aryzon SDK, Unity and XCode. Fig. 10 Likert scale on incentivisation to reduce waste 123 Augmented Human Research (2022) 7:2 Page 11 of 12 2 Fig. 11 Measure of food waste provided During the evaluation, 60% of the survey participants Aryzon), but in future, this platform could be switched to agreed that the app helped with understanding the level of an online based platform, that gathers data online and waste. All participants agreed that the app clearly visu- updates real time. Additional features could include more alises which categories are wasted the most. Notably realistic models, a greater diversity in products, a menu, almost no participants disagreed and no participant and clearer textual information. As for the headset, most strongly disagreed with these statements. Another key users liked reacted positively, but it is strongly suggested to ﬁnding is that every participant indicated they were sur- make the app usable both with, and without a headset for prised by the amount wasted and answered that the waste practicality. The 3D visualisation of the data was mostly was more than they expected. When asked what users liked favoured by most users and seems to be a viable repre- the least, 33% said the textual information. This may be a sentation of the food waste data. Especially when com- point to improve upon in further iterations of the app. paring this to more conventional methods such as pie charts Furthermore, when directly asked ‘What do you think the or static images. mobile application should improve on?’ participants had The research about food waste and AR is still in its the opportunity to provide qualitative response. Replies infancy and more research needs to be done to be able to included to provide ‘a more clear presentation of the text, draw conclusions on the matter. However, the technology and higher visual qualities of the objects’, to add ‘more of a is viable and may contribute to the reduction of food waste menu, maybe different restaurants’, to offer a ‘clearer in the restaurant business. Further investigation is neces- distinction between types of food’, to ‘make the food more sary to expand on the technology and to bring it to con- 3D shaped’, and ﬁnally the ‘Ability to select other sumers and households as well. The current application has weeks/days to see the difference’. limitations with representing the data and intuitiveness. With regards to the immersion, all users felt immersed This will need some improvements for larger scale uptake in the visualisation but all mentioned that the immersion and use in a business setting. Limitations also include the can be improved upon. One suggestion was to increase the target evaluation group, as the focus of the survey was on polygon count on the 3D models and another suggestion one select group of users, who are familiar with smart was to use more models in the same art style. The overall phone technologies. Other less technologically capable experience of the app was rated a 4/5 by the testers. 42% users may provide different responses. In future work, the would rather use the application without the headset. Other authors will expand the study to wider user groups to suggestions for improvements in the application were for a collate more detailed opinions and reﬂections. clearer representation of the text. Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s41133- 022-00057-7. Conclusion Declarations To conclude, within the constraints of the limited testing process, the use of an AR application to help reduce food Conflict of interest The authors declare no conflict of interest. waste is promising. As this research was limited in par- ticipants due to the Covid-19 restrictions, no real reduction Open Access This article is licensed under a Creative Commons in food waste could be measured to accompany the study, Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as but this will be considered in future work. However, some long as you give appropriate credit to the original author(s) and the suggestions that can be made are as follows. In this case, source, provide a link to the Creative Commons licence, and indicate the most suitable AR platform was a premade SDK (from 123 2 Page 12 of 12 Augmented Human Research (2022) 7:2 if changes were made. The images or other third party material in this 11. Rodrigues JM, Ramos CM, Pereira JA, Sardo JD, Cardoso PJ article are included in the article’s Creative Commons licence, unless (2019) Mobile ﬁve senses augmented reality system: technology indicated otherwise in a credit line to the material. If material is not acceptance study. IEEE Access 7:163022–163033 included in the article’s Creative Commons licence and your intended 12. Fuchs K, Haldimann M, Grundmann T, Fleisch E (2020) Sup- use is not permitted by statutory regulation or exceeds the permitted porting food choices in the Internet of People: Automatic use, you will need to obtain permission directly from the copyright detection of diet-related activities and display of real-time holder. To view a copy of this licence, visit http://creativecommons. interventions via mixed reality headsets. Future Generations org/licenses/by/4.0/. Computer Systems 113:343–362 13. Li L, Qiao X, Lu Q, Ren P, Lin R (2020) Rendering optimization for mobile web 3D based on animation data separation and on- demand loading. IEEE Access 8:88474–88486 References 14. Fujimoto Y (2018) Projection mapping for enhancing the per- ceived deliciousness of food. IEEE Access 6:59975–59985 1. Voedingscentrum, (2021) Voedselverspilling. Voedingscentrum, 15. Gong L, Fast-Berglund A, Johansson B (2021) A framework for Den Hague extended reality system development in manufacturing. IEEE 2. Scherhaufer S, Moates G, Hartikainen H, Waldron K, Obersteiner Access 9:24796–24813 G (2018) Environmental impacts of food waste in Europe. Waste 16. Qiao X, Ren P, Dustdar S, Liu L, Ma H, Chen J (2019) Web AR: Manage 77:98–113 A promising future for mobile augmented reality—State of the 3. Pinto RS, Dos Santos Pinto RM, Silva Melo FF, Campos SS, art, challenges, and insights. Proc IEEE 107(4):651–666 Santos Pinto CMDS (2018) A simple awareness campaign to 17. M. G. Guillen, J. Quist and J. Hamari (2021) Gamiﬁcation of promote food waste reduction in a University canteen. Waste backcasting for sustainability: the development of the Gameful Manage 76:28–38 Backcasting Framework (GAMEBACK),’’ Journal of Cleaner 4. Rauschnabel PA, Felix R, Hinsch C (2019) Augmented reality Production, 126609 marketing: How mobile AR-apps can improve brands through 18. Hoppenstedt B, Probst T, Reichert M (2019) Applicability of inspiration. J Retail Consum Serv 49:43–53 immersive analytics in mixed reality: Usability study. IEEE 5. J. Barrow, C. Forker, A. Sands, D. O’Hare and W. Hurst (2019) Access 7:71921–71932 Augmented reality for enhancing life science education,’’ in 19. Shea R, Fu D, Sun A, Cai C, Ma X (2017) Location-based VISUAL, Venice augmented reality with pervasive smartphone sensors: Inside and 6. Nicholson-Cole SA (2005) Representing climate change futures: beyond Pokemon Go! IEEE Access 5:9619–9631 a critique on the use of images for visual communication. Comput ´ ˜ ´ 20. Ibanez MB, Di-Serio A, Villaran-Molina D, Delgado-Kloos C Environ Urban Syst 29(3):255–273 (2015) Support for augmented reality simulation systems: The 7. Nechypurenko PP, Stoliarenko VG, Starova TV (2020) Devel- effects of scaffolding on learning outcomes and behavior pat- opment and implementation of educational resources in chem- terns. IEEE Trans Learn Technol 9(1):46–56 istry with elements of augmented reality’’, in Augmented Reality 21. B. Song and R. Qiu, (2020) he Inﬂuence of Digital Virtual in Education. Kryvyi Rih, Ukraine Technology on Contemporary College Students’ Ideological and 8. Carl B, Bopp M, Saß B, Voellger B, Nimsky C (2019) Imple- Political Education,’’ IEEE Access, 3020167 mentation of augmented reality support in spine surgery. Eur 22. Kim H, Lee M, Kim GJ, Hwang JI (2021) The impacts of visual Spine J 28(7):1697–1711 effects on user perception with a virtual human in augmented 9. Zhou Y, Luo H, Yang Y (2017) Implementation of augmented reality conﬂict situations. IEEE Access 9:35300–35312 reality for segment displacement inspection during tunneling 23. Koutitas G, Vyas S, Vyas C, Jadon SS, Koutsopoulos I (2020) construction. Autom Constr 82:112–121 Practical methods for efﬁcient resource utilization in augmented 10. Marto A, Melo M, Gonc¸alves A, Bessa M (2020) Multisensory reality services. IEEE Access 8:220263–220273 augmented reality in cultural heritage: impact of different stimuli on presence, enjoyment, knowledge and value of the experience. Publisher’s Note Springer Nature remains neutral with regard to IEEE Access 8:193744–193756 jurisdictional claims in published maps and institutional afﬁliations. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Augmented Human Research Springer Journals http://www.deepdyve.com/lp/springer-journals/harnessing-augmented-reality-for-increasing-the-awareness-of-food-EGcJzBDnMn

Loading next page...

Publisher: Springer Journals
Copyright: Copyright © The Author(s) 2022
ISSN: 2365-4317
eISSN: 2365-4325
DOI: 10.1007/s41133-022-00057-7
Publisher site: See Article on Publisher Site

Abstract

Food waste is a signiﬁcant challenge, and our societal behaviours play a role in the amount of food items discarded. Thus, an effective method to inform consumers about high wastage patterns may help reduce the amount thrown away. This research investigates how Augmented Reality can be harnessed to enlighten consumers and work towards addressing high food waste patterns. Yet research on this topic is still very much in its infancy. To pursue this solution, food behaviour data are employed to provide an insight into how much is wasted from 9 catering industry locations in the Netherlands. An Augmented Reality application is developed, where models of food are projected onto real-world environments to provide scale on waste over a 7-day period. A quantitative evaluation of higher-education attendees demonstrated the approach has potential to incentivise reduction in waste. Keywords Augmented reality Food waste Catering industry Consumer Introduction Understanding the volume of waste is a particular challenge for consumers, and digital technologies, such as Food waste is a signiﬁcant problem in the current economy. Augmented Reality (AR), could provide a supportive A considerable volume of food is thrown away or wasted. metric to increase awareness with the potential to inspire In the Netherlands, about 9.5% of all bought food is wasted consumers positively [4] by providing a visual aid to yearly; this equates to 34.3 kg of solid foods [1] and understand the amount of waste produced. AR is a suit- equivalent to 154 kg carbon dioxide (CO ) per person per able technology for this purpose, as it is becoming more year. Furthermore, this accounts for 8–10% of all the accessible and widespread, largely due to the capabilities greenhouse gasses emitted by the Netherlands [1]. In wider of smart phones. Because of this, in the current environ- Europe, about 186 Mt CO -eq is attributed to food waste ment, almost every consumer has an AR-capable device, [2]. This has resulted in food campaigns, such as the clean making the technology an ideal solution for visual com- dish, clean conscience! initiative in Portugal, to help munication. For instance, each smart phone is equipped increase awareness for food waste behaviours [3]. How- with a screen, camera and hardware, making the devices ever, in general, as conﬁrmed by UN Sustainable Devel- suitable for running modern-day AR applications. Fur- opment Goal SDG 12.3, more solutions are required to thermore, AR is already used regularly in education and address the pattern of waste. communicative applications. For example, according to [5], AR is a suitable method to deliver educational material due to its ability to deliver a learning experience that & William Hurst combines experiential, kinaesthetic and didactic learning. will.hurst@wur.nl Similarly, in a study by Nicholson-Cole et al. [6], visual Dolf Honee communication showed a positive impact on the perceived dolf.honee@wur.nl importance of the prominent subject of climate change. Antonius Johannus Luttikhold Thus, demonstrating that users are becoming familiar with thomas@wastewatchers.eu AR applications and that a visual element is an ideal metric Information Technology Group, Wageningen University and for communication. Research, Wageningen, The Netherlands Wastewatchers, Utrecht, The Netherlands 123 2 Page 2 of 12 Augmented Human Research (2022) 7:2 It may be challenging to grasp what it means to waste Search Strategy 8–10% of bought food, but a visual representation of this may have a clearer impact. The theory is that, through the The focus of the related work is on two digital libraries use of an AR experience, consumers may be more inclined IEEE Xplore and Science Direct. As AR is an emerging to both understand and work towards a reduction of food technology with continuous developments, only literature waste behaviours. In order to address this, the study from the last ﬁve years (2016–2021) is considered in this investigates suitable AR approaches (from related works) study. A survey of the literature landscape demonstrated for the creation a food waste application. An in-depth that a signiﬁcant amount of AR-based articles are confer- analysis of related work is conducted using a snowballing ence based. Therefore, the targeted sources include jour- technique, in which only open-access articles are consid- nals, conferences and workshops. For both libraries, a ered for repeatability of the investigation. Based on the focused general search query is used to identify papers ﬁndings, a prototype AR application for food waste visu- speciﬁcally addressing food waste and AR. The search alisation is designed and developed using the Aryzon AR catered for possible variations in the key words. This Software Development Kit (SDK). At the time of writing resulted in the following generalised search query, where this article, Aryzon is an emerging technology and yet to be 458 papers were identiﬁed (75 from Science Direct and 383 employed within the food waste visualisation domain. from IEEE Xplore). Therefore, the technology is not found within the related ðAugmented Reality OR Mixed Reality OR Immersive OR works discussion but is an ideal solution due to the inte- Virtual OR 3Dor XRÞ ANDðÞ Food waste OR Food loss gration with Unity game engine and that it caters for both ð1Þ screen-based and wearable AR deployment. Evaluation of the AR application involved participants attending a (1) is an abstracted example of the search string, as vari- higher-education institute, a similar approach to the work ations of Augmented and Mixed reality (e.g. AR, XR, MR) by Pinto et al. in [2] for the promotion of food waste were included in the full search. Initially, the search string reduction. However, access to this participant group was had a broad scope to ensure no potentially relevant research limited due to social distancing measures at the time of articles are missed. From the number of papers, the most testing. relevant ones were ﬁltered using the selection criteria The remainder of the article is as follows. ‘‘Related presented in Table 1. The selection criteria are applied Work’’ section provides a background discussion and lit- manually by reading the title and abstract of the studies. erature review of the current use of AR for food waste This reduced down the number of articles down to 15 (a applications (at the time of writing this, there are relatively full list is presented in Table 2). An immediate observation few articles on this topic). ‘‘Materials and Methods’’ sec- is that there are few articles available that combine both tion discusses the methodology adopted for the develop- AR and food waste, demonstrating that the work presented ment of a AR application, and ‘‘Results’’ section provides in the article is within a potential research niche. In the and overview of the implementation. The article ﬁndings following subsection, the ﬁndings from these 15 papers are are discussed in ‘‘Discussion’’ section, and conclusions are discussed. presented in ‘‘Conclusion’’ section. Findings Related Work Within the 15 papers, differing development platforms are employed for the construction of the AR applications. AR has a broad scope of applications, examples include However, 5 studies did not state which platform they used teaching chemistry in higher education [7], support for (mostly because the investigations within the 5 articles are spinal surgery [8] and assisting with building tunnels in applied to the general implementation of AR). The most construction [9], to name but a few. As discussed, the focus used platforms are self-created; an approach adopted by of this article is on AR in the food waste domain, which is a 4/15 articles, for example, the SensiMAR prototype by research concept that is very much in its infancy. There- Marto et al., [10] and the M5SAR by Rodrigues et al. [11]. fore, in order to design and develop an effective AR This customised approach is a suitable solution to work solution, ﬁrst an investigation is required into related works with, as the tools are speciﬁcally made for their intended in order to ascertain the appropriate features to employ and goal. As Marto et al. [10] discuss, the use of a custom-built obstacles to avoid during the design and implementation of AR technology is necessary because no comparable tech- the solution. In this section, an overview of the ﬁeld of nologies were available with the same features. However, a interest is provided. limitation of this method is that it has signiﬁcant 123 Augmented Human Research (2022) 7:2 Page 3 of 12 2 Table 1 Study selection criteria No Criterion SC1 Papers without full text available SC2 Papers not written in English SC3 Duplicate publication from multiple sources SC4 Papers do not relate to AR SC5 Papers are not directly related to this study (i.e., not applicable to an AR app for food waste) The range of the categories indicates how ﬂexible Table 2 Study selection criteria AR is for deployment in varied settings. Table 3 dis- Fuchs et al. [12] 2020 Li et al. [13] 2020 plays the full ﬁndings and identiﬁed obstacles from the Fujimoto [14] 2018 Marto et al. [10] 2020 selected studies. Crucially, of the 15 papers analysed, Gong et al. [15] 2021 Qiao et al. [16] 2019 none is in the domain of food waste speciﬁcally, but Guillen et al. [17] 2021 Rodrigues et al. [11] 2019 there were some synergies; for example, Fujimoto et al. Hoppenstedt et al. [18] 2019 Shea et al. [19] 2017 [14] discuss the perceived deliciousness of food and Iba´n˜ez et al. [20] 2016 Song et al., [21] 2020 how this can be increased by chroma and colour Kim et al. [22] 2021 Zhou et al. [9] 2020 modiﬁcations. This would be in line with the ﬁndings in Koutitas et al. [23] 2020 Marto et al. [10] where a demonstration is provided that additional sensory stimuli may impact the experience of a user within AR. In terms of engagement, Hoppenstedt et al. [18] development costs due to the high levels of customisation. discuss that immersive analytics by means of AR are a 3/15 studies used Web AR as their main technology for suitable replacement for 2D representations of data and creating an application. Web AR is a ﬂexible approach, as may be even better for retention. Furthermore, Fuchs no app has to be installed and is compatible with almost et al. [12] highlight how the visual element of AR has any device that is AR-capable. A disadvantage of this an impact when users make a choice about which food method is that, at the time of this investigation, Web AR they will buy. For the design of any future food waste does not have a broad range features compared to other applications, this may suggest altering the 3D models custom-built AR platforms, such as AR Core. Other plat- with chroma and partial-colour modiﬁcations to help forms that are used in the articles include Unity AR, make the models look more appetising and incentivise Vuforia, and Magic Leap, as displayed in Fig. 1a. The users to waste less of these foods. Therefore, in the application domains of the 15 papers can be grouped into 8 application detailed in ‘‘Materials and Methods’’ section, categories, cultural, education, entertainment, food and the data will be presented through a combination of 3D diet, manufacturing, medical, sustainability and technol- models and 2D textual elements that further elaborate ogy, as in Fig. 1b. on the 3D models. In this way, the user is engaged by Fig. 1 a AR methods used b Research Domains 123 2 Page 4 of 12 Augmented Human Research (2022) 7:2 Table 3 Related work summary Study ID Findings Obstacles AR Research Features Methods Domain Fuchs et al Mixed reality (XR) can support healthier XR interventions are a promising Unity AR Food and Diet Wearable food choices ﬁeld of research foundation device ? App Fujimoto Digital representation of food may relate Limited number of food types Web AR Food and Diet No wearable et al to its perceived deliciousness considered in the investigation device, no app Gong et al Virtual reality (VR) may have a Application depends heavily on Vuforia and Manufacturing Wearable suitable user acceptance with the user acceptance of the VR Unity device ? App appropriate framework hardware Guillen et al Gamiﬁcation may offer user engagement There is a ﬁne line between Not Sustainability No wearable beneﬁts within sustainability subjects gamiﬁcation and game-based speciﬁed device ? App activities – and the long-term impact is inconclusive Hoppenstedt AR analytics may be a A wearable headset may be a Not Medical Wearable et al suitable replacement for 2D data– A possible stressor speciﬁed device ? App wearable headset may improve user engagement Iba´n˜ez et al AR can support with learning and Further study is necessary Own Education Wearable information retention in education platform device ? App Kim et al User perception in AR may depend on Perceived immersion may be Magic leap Technology Wearable interaction between virtual and physical disrupted when virtual objects device ? App overlap with physical objects Koutitas et al Reduce both computational resources and For best performance, caching Not Technology Wearable the induced network trafﬁc, by caching algorithm requires detailed speciﬁed device ? App images information about the user and object locations Li et al Web-based mobile AR and VR loading The number of animation tracks Web AR Technology No wearable times and experience can be improved and the size of the models device—No through on demand loading of 3D affect the optimisation of this app models method Marto et al Combination of sensory stimuli (audio, Impact of sensory stimuli on Own Cultural Wearable visual and smell) may improve overall experience changes platform device ? App enjoyment and knowledge retention in widely combined with other AR stimuli Qiao et al Web AR has a promising future for Depends highly on internet Web AR Technology No wearable mobile AR speeds -Web based AR is still device—No in its infancy stage app Rodrigues Social inﬂuence, effort expectancy, and Research only limited to one Own Cultural Wearable et al facilitating conditions are important in type of AR technology platform device ? App user acceptation in AR technology Shea et al Smartphones and their pervasive sensing AR apps with heavy pervasive Own Entertainment No wearable possibilities can result in successful AR sensing consumes signiﬁcant platform device ? App applications battery power Song & Qiu AR teaching methods may improve the Only investigated in political Not Education No wearable learning progress of contemporary and ideological education – speciﬁed device ? App college students Not a replacement for conventional education Zhou et al Different display technologies may affect In terms of more complex Not Technology Wearable users’ visual comfort, interaction learning activities and long- speciﬁed device ? App experience, learning experience, and term learning further study outcome in hands-on learning needs to be done the stereoscopic images, where speciﬁc information (that Regarding the hardware, one item of note is that a would be difﬁcult to represent in a 3D model) is pro- wearable device may be a possible stressor for users and vided through text overlay. this should be factored in when developing an AR 123 Augmented Human Research (2022) 7:2 Page 5 of 12 2 application for food waste awareness. The evaluation is present or the user taps to a location on the screen, the process in this article takes note of this, by offering a application will gather the required data from an internal testing process that makes use of both a screen and headset, database and start to project the 3D visualizations in AR. allowing the user to choose. This approach also aligns with When the user is ﬁnished, they have the option to close the 9/15 papers, which investigated an application and a app or start a new QR code scan. wearable headset combination, whereas 3/15 made use of only an application and no wearable headset, and another Data 3/15 made use of no installed application and no headset. Yet, according to Fuchs et al. (2020), headsets help to To ensure that the application and visualisation produced achieve high presence and immersions for the users. are based on a real-world scenario, a dataset containing Moreover, a wearable headset may improve user engage- 40,000 entrees from nine different locations in the ment [18]. An installed application has the advantage that it Netherlands from 01 August 2020 until 29 March can have 3D models pre-installed and allow for more 2021–2020 is employed (due to the signed agreement with complex features (an example of this can be the case of the data use, the locations cannot be documented in the Poke´mon Go! [19]). article to maintain privacy). At the time of writing this article, no AR technologies Within the data, the location, date, price and wasted within the articles evaluated have been developed speciﬁ- item by category are detailed. Artefacts (measurement cally for food waste. Based on other approaches in different errors) were ﬁltered from the raw data through consultation categories, the ﬁndings indicate that the most suitable op- with the data provider. An example of the data can be seen tions for the design of an AR app for food waste would be below in Table 4 (where labels A, B and C are employed to to create an app using a custom-built SDK (e.g. Aryzon). maintain the anonymity of the locations). This would cater for ﬂexibility, both for the design and When comparing waste measurements within the entire implementation process, as much of the framework and dataset, as in Fig. 3, bread and snacks are visibly the code libraries are already laid out. highest amongst the categories of left-over waste in all locations. However, bread and snacks are also the most sold categories. Materials and Methods Therefore, the percentage wasted compared to the units (sales) prepared by the cafeterias is factored in. The rela- The design for an AR application for visualising food tive waste is calculated by dividing the total units (sales) waste is put forward in this section. The approach is based wasted per category by the total units prepared per cate- on the ﬁndings from the literature review in ‘‘Related gory. The formula is used to calculate this, as follows in Work’’ section. Speciﬁcally, the design framework and (2): data used for the application are presented. The section is Total units of waste Lative waste per prepared unit ¼ 100% concluded with an overview of the hardware and 3D con- Total units prepared struction process. ð2Þ It becomes clear that bread and snacks are not the largest Design producers of waste compared to their amounts produced. This is evident in Fig. 4a and b, where, with almost 30%, A business process model and notation diagram is created grains are the highest waste producer relative to units below in Fig. 2 to represent how the app will function. The prepared. Aryzon SDK and the Unity platform are selected for the development of the application. Unity allows for custom For example, relatively fewer grains are purchased by the cafes and, thus, the total waste is not as signiﬁcant as, creation within the application, whereas the Aryzon SDK for example, bread or soup. Waste data are also split into supports the deployment process. In Fig. 2, when the user categories per days of the week. Tuesdays, Wednesdays, opens the application, they land on the home screen of the Thursdays and Fridays are very similar in their waste application. From here, users have the option to scan a QR distribution. code (if necessary) and open the camera or close the In Fig. 5, boxplots are used to compare waste distribu- application. A QR code is the most suitable approach for tion between days of the week, with food category on the interaction in a dynamic environment, such as a kitchen. x-axis and units wasted on the y-axis. A difference is Other approaches include using marker-less, which is the evident between the ﬁrst day of the week, last day of the approach adopted for the trial in this article. This is because week and the weekend. On Friday, the amount of meal the Aryzon technology is able to track ﬂat surfaces with a suitable accuracy for the evaluation. When a valid QR code salads wasted is often higher. Monday sees more waste in 123 2 Page 6 of 12 Augmented Human Research (2022) 7:2 Fig. 2 Framework Diagram for the AR application Table 4 Example data Date Location ID Product ID Amount Produced Amount Wasted Price Day Category (Euros) 29–03-2021 A 278 1 0 1,72 1 Snacks 29–03-2021 A 531 2 1 1,15 1 Snacks 26–03-2021 A 927 6 0 5,31 5 Hot meal 26–03-2021 B 277 38 5 2,03 5 Snacks 26–03-2021 C 2349 8 7 0,91 5 Dairy Visualisation In order to represent this information within an AR appli- cation, the visualisation will be based on 3D models ren- dered in the Unity engine, where a prototype is developed with the Aryzon AR Studio SDK. The AR tracking from Aryzon in the application is able to recognise ﬂat surfaces, without any major loading interruptions or errors. To minimise processing power demand, instead of rendering each amount separately, waste amounts will be categorized in 3D models that represent different amounts. For the Bread category for example, this means there will be a model for 1 unit of waste (e.g. one loaf of bread, or one snack item etc.), a model for 15 units of waste and a model for 25 units of waste. Conceptual examples are given for Fig. 3 Food waste by category for 9 locations bread, drinks and fruits and vegetables in Table 5, and displayed in Unity in Fig. 6. Whilst drinks are include for the soup category. On Sunday, it is very apparent that the the conceptualisation, only food is involved in the ﬁnal category Ingredient Level is wasted more compared to application prototype. other days of the week. A clear visualisation is the key feature of the applica- tion. A user should be able to identify what the 123 Augmented Human Research (2022) 7:2 Page 7 of 12 2 Fig. 4 a Food waste per category. August 2020 – March 2021 b Relative waste per prepared unit per category. August 2020 – March 2021 visualisation represents and have an idea how much it is. It is, therefore, signiﬁcant to make sure the 3D models are projected in realistic dimensions. Along with the models, additional data (in the form of text) are provided to support the visualisations. To recreate an appropriate level of realism, all the models have realistic textures that resemble the real-world equivalent. Hardware For the application to work, an AR compatible phone is employed. This means for IOS devices running IOS 11 or later and for Android devices running Android 7.0 or later. Furthermore, the application is developed for use both with and without the Aryzon headset as displayed in Fig. 7. Fig. 5 Boxplot of waste by category for 9 locations Table 5 Waste units Amount (units) 3D Model (Bread) 3D Model (Drinks) 3D Model (Fruits & Vegetables) represented by 3D models 123 2 Page 8 of 12 Augmented Human Research (2022) 7:2 Fig. 6 First trials with 3D models in unity Fig. 7 Aryzon’s cardboard headset a Side view b Front view into visor The Aryzon headset is constructed from cardboard, 19 quantitative involving use of the Likert scale and 4 which houses a mobile phone below the eye-level. Pro- qualitative where text could be inserted (an overview of the jections are created using an optical see-through approach, questionnaire is provided in Online Appendix A). where mirrors combine 3D model with views of the The participants of this test were characterized as fol- physical world. This means the real-world image is as lows: 19 testers, all of them experts with mobile phones natural as possible. The headset has a head strap for a and with more than three years of background experience hands-free experience. with smartphones and applications (aged between 18 and 24 years with, at least, a Bachelor’s Degree). Due to timeframe of the research and the current status of Covid- Results 19, the test group was limited in size and diversity. In the tests, users could install the Android Package (APK) on Testing is performed in the kitchen of the Leeuwenborch their Android smartphone or try the application out the IOS building from Wageningen University campus to evaluate smartphone provided by the authors. Users were introduced the real-world application (Fig. 8). 25 testers (students of to the topic and after the demonstration were explained higher education) were randomly approached on premise to what the visualizations represented. The visualisation try out the application. The process involved asking users provided for the evaluation only concerned food waste and to use the AR application, view the 3D objects and read the not drink. The inclusion of drink will be considered in text through means of viewing the application on a mobile future testing. phone screen. Out of these 25, 19 were available (access to a larger survey base was severely limited by the Covid-19 Features restrictions). 19 users also tried the AR application by using the headset for comparison with the screen-based When asked about features, participants got the choice to approach. Users were then asked to complete 23 questions, select every feature they favoured the most; no limit was 123 Augmented Human Research (2022) 7:2 Page 9 of 12 2 Fig. 8 AR application trial set to the number they are able to select (Fig. 9). 21% of observation was that the intuitiveness was chosen as the participants favoured the ease of use of the application the least as a favourite feature. When asked what users liked most. This may be a positive indication for the imple- the least, 33% said the textual information (Fig. 8). This mentation of the application within the catering business, may be a point to improve upon in further iterations of the as business professionals often may not want to spend time application. It is also notable that 25% of the survey par- learning a new application that keeps them from their ticipants chose ‘none’ as their most disliked feature. work. Second, with 18%, both the 3D models and the AR One user suggested to add CO emission information tracking were popular features. Overall, the most liked and another user suggested to add a menu. With regards to features were fairly widespread; where one apparent the immersion, all users felt immersed in the visualisation Fig. 9 Most liked/disliked features 123 2 Page 10 of 12 Augmented Human Research (2022) 7:2 but all mentioned that the immersion can be improved methods of data visualisation, all users preferred the 3D upon. A further suggestion was to increase the polygon visualisation of the application. Particularly for textual count on the 3D models (which would add more realism to reports and static images, this method for visualisation of the models) and another suggestion was to use more data may be preferred by about 70% of the testers. models in the same art style (e.g. to include drinks). The overall experience of the app was rated a 4/5 by the testers. The experience with the headset was the lowest with 3.4 Discussion out of 5 points. As discussed, food waste is a signiﬁcant issue in society, Food Waste and prominent particularly within the current catering industry; demonstrated by the data overview in ‘‘Materials When asked about this application incentivising for and Methods’’ section. Although 2D graphical representa- reducing food waste, 21% of the participants gave it a 5/5 tions may give an indication about how much food waste on a Likert scale and 37% gave the incentivization a 4/5. there is, a 3D representation would help with provide a About 16% rated this with a 3/5. 26% thought the appli- level of scope. AR has a broad scope of uses, but relatively cation was not very likely to have an impact. However, no few apps exist in the domain of food waste. In fact, to the users ﬁlled in the 1/5 option (Fig. 10). Notably, 60% of the best of our knowledge, at the time of writing this article no users were surprised by the information the app gave and applications speciﬁcally for AR with food waste exist. Yet, agreed that the application helped with understanding the AR applications in other domains have shown to help with level of waste. All participants agreed that the app clearly information retention and engagement, particularly in visualises which categories are wasted the most. education. From the literature review, it became clear that the Data Visualisation research in food waste and AR is still in its infancy. Most domains of the researched AR papers are within the tech- In terms of the usability to view the data, 42% would rather nology category, with 33%. 60% of papers made use of use the application without the headset. 26% would like to both a wearable device and an application. 27% of papers use the application with the headset and about 32% would made use of their own platform and 33% of the papers did prefer a hybrid approach (both with and without the not specify which AR platform they used. The rest of the headset), as displayed in Fig. 11. researched papers all used different platforms. One item of Regarding the question as to whether the visualization note is that a wearable device may be a possible stressor for provides a measure of how much food is wasted resulted in users. varied feedback. The most users (37%) gave this a 4/5 on A variety of platforms also exist for building an AR the Likert scale. The rest of the answers were divided application, each comes with their own advantages and between 2/5, 3/5 and 5/5. However, all participants indi- disadvantages. The most suitable option for this research cated that they were surprised by the amount wasted and seemed to be to create an app with a wearable headset and answered that the waste was more than they expected. The a custom-built platform. However, the use of an existing realism of the models was rated a 4/5, with some users SDK (e.g. Aryzon) caters for some ﬂexibility with indicating that the realism can be improved to improve the designing, as well as a relative quick development because overall immersion. Other suggestions for improvements in the framework is already laid out. the app were more diversiﬁcation between food categories Due to limited applications known about both subjects, and a clearer representation of the text. Compared to other this research is an opportunity to learn more about how AR can inﬂuence food waste attitudes or behaviours. The data are presented through a combination of 3D models and 2D textual elements that further elaborate on the 3D models. In this way, the user is engaged by the stereoscopic images and can get more speciﬁc information from text that would be difﬁcult to represent in a 3D model. During the devel- opment, it was chosen to use models that represent multiple units of food. A ﬁrst prototype was made and tested in various situations. Later on, models that represent all food waste from one week were implemented along with textual information overlay. The software used for developing the application was Aryzon SDK, Unity and XCode. Fig. 10 Likert scale on incentivisation to reduce waste 123 Augmented Human Research (2022) 7:2 Page 11 of 12 2 Fig. 11 Measure of food waste provided During the evaluation, 60% of the survey participants Aryzon), but in future, this platform could be switched to agreed that the app helped with understanding the level of an online based platform, that gathers data online and waste. All participants agreed that the app clearly visu- updates real time. Additional features could include more alises which categories are wasted the most. Notably realistic models, a greater diversity in products, a menu, almost no participants disagreed and no participant and clearer textual information. As for the headset, most strongly disagreed with these statements. Another key users liked reacted positively, but it is strongly suggested to ﬁnding is that every participant indicated they were sur- make the app usable both with, and without a headset for prised by the amount wasted and answered that the waste practicality. The 3D visualisation of the data was mostly was more than they expected. When asked what users liked favoured by most users and seems to be a viable repre- the least, 33% said the textual information. This may be a sentation of the food waste data. Especially when com- point to improve upon in further iterations of the app. paring this to more conventional methods such as pie charts Furthermore, when directly asked ‘What do you think the or static images. mobile application should improve on?’ participants had The research about food waste and AR is still in its the opportunity to provide qualitative response. Replies infancy and more research needs to be done to be able to included to provide ‘a more clear presentation of the text, draw conclusions on the matter. However, the technology and higher visual qualities of the objects’, to add ‘more of a is viable and may contribute to the reduction of food waste menu, maybe different restaurants’, to offer a ‘clearer in the restaurant business. Further investigation is neces- distinction between types of food’, to ‘make the food more sary to expand on the technology and to bring it to con- 3D shaped’, and ﬁnally the ‘Ability to select other sumers and households as well. The current application has weeks/days to see the difference’. limitations with representing the data and intuitiveness. With regards to the immersion, all users felt immersed This will need some improvements for larger scale uptake in the visualisation but all mentioned that the immersion and use in a business setting. Limitations also include the can be improved upon. One suggestion was to increase the target evaluation group, as the focus of the survey was on polygon count on the 3D models and another suggestion one select group of users, who are familiar with smart was to use more models in the same art style. The overall phone technologies. Other less technologically capable experience of the app was rated a 4/5 by the testers. 42% users may provide different responses. In future work, the would rather use the application without the headset. Other authors will expand the study to wider user groups to suggestions for improvements in the application were for a collate more detailed opinions and reﬂections. clearer representation of the text. Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s41133- 022-00057-7. Conclusion Declarations To conclude, within the constraints of the limited testing process, the use of an AR application to help reduce food Conflict of interest The authors declare no conflict of interest. waste is promising. As this research was limited in par- ticipants due to the Covid-19 restrictions, no real reduction Open Access This article is licensed under a Creative Commons in food waste could be measured to accompany the study, Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as but this will be considered in future work. However, some long as you give appropriate credit to the original author(s) and the suggestions that can be made are as follows. In this case, source, provide a link to the Creative Commons licence, and indicate the most suitable AR platform was a premade SDK (from 123 2 Page 12 of 12 Augmented Human Research (2022) 7:2 if changes were made. The images or other third party material in this 11. Rodrigues JM, Ramos CM, Pereira JA, Sardo JD, Cardoso PJ article are included in the article’s Creative Commons licence, unless (2019) Mobile ﬁve senses augmented reality system: technology indicated otherwise in a credit line to the material. If material is not acceptance study. IEEE Access 7:163022–163033 included in the article’s Creative Commons licence and your intended 12. Fuchs K, Haldimann M, Grundmann T, Fleisch E (2020) Sup- use is not permitted by statutory regulation or exceeds the permitted porting food choices in the Internet of People: Automatic use, you will need to obtain permission directly from the copyright detection of diet-related activities and display of real-time holder. To view a copy of this licence, visit http://creativecommons. interventions via mixed reality headsets. Future Generations org/licenses/by/4.0/. Computer Systems 113:343–362 13. Li L, Qiao X, Lu Q, Ren P, Lin R (2020) Rendering optimization for mobile web 3D based on animation data separation and on- demand loading. IEEE Access 8:88474–88486 References 14. Fujimoto Y (2018) Projection mapping for enhancing the per- ceived deliciousness of food. IEEE Access 6:59975–59985 1. Voedingscentrum, (2021) Voedselverspilling. Voedingscentrum, 15. Gong L, Fast-Berglund A, Johansson B (2021) A framework for Den Hague extended reality system development in manufacturing. IEEE 2. Scherhaufer S, Moates G, Hartikainen H, Waldron K, Obersteiner Access 9:24796–24813 G (2018) Environmental impacts of food waste in Europe. Waste 16. Qiao X, Ren P, Dustdar S, Liu L, Ma H, Chen J (2019) Web AR: Manage 77:98–113 A promising future for mobile augmented reality—State of the 3. Pinto RS, Dos Santos Pinto RM, Silva Melo FF, Campos SS, art, challenges, and insights. Proc IEEE 107(4):651–666 Santos Pinto CMDS (2018) A simple awareness campaign to 17. M. G. Guillen, J. Quist and J. Hamari (2021) Gamiﬁcation of promote food waste reduction in a University canteen. Waste backcasting for sustainability: the development of the Gameful Manage 76:28–38 Backcasting Framework (GAMEBACK),’’ Journal of Cleaner 4. Rauschnabel PA, Felix R, Hinsch C (2019) Augmented reality Production, 126609 marketing: How mobile AR-apps can improve brands through 18. Hoppenstedt B, Probst T, Reichert M (2019) Applicability of inspiration. J Retail Consum Serv 49:43–53 immersive analytics in mixed reality: Usability study. IEEE 5. J. Barrow, C. Forker, A. Sands, D. O’Hare and W. Hurst (2019) Access 7:71921–71932 Augmented reality for enhancing life science education,’’ in 19. Shea R, Fu D, Sun A, Cai C, Ma X (2017) Location-based VISUAL, Venice augmented reality with pervasive smartphone sensors: Inside and 6. Nicholson-Cole SA (2005) Representing climate change futures: beyond Pokemon Go! IEEE Access 5:9619–9631 a critique on the use of images for visual communication. Comput ´ ˜ ´ 20. Ibanez MB, Di-Serio A, Villaran-Molina D, Delgado-Kloos C Environ Urban Syst 29(3):255–273 (2015) Support for augmented reality simulation systems: The 7. Nechypurenko PP, Stoliarenko VG, Starova TV (2020) Devel- effects of scaffolding on learning outcomes and behavior pat- opment and implementation of educational resources in chem- terns. IEEE Trans Learn Technol 9(1):46–56 istry with elements of augmented reality’’, in Augmented Reality 21. B. Song and R. Qiu, (2020) he Inﬂuence of Digital Virtual in Education. Kryvyi Rih, Ukraine Technology on Contemporary College Students’ Ideological and 8. Carl B, Bopp M, Saß B, Voellger B, Nimsky C (2019) Imple- Political Education,’’ IEEE Access, 3020167 mentation of augmented reality support in spine surgery. Eur 22. Kim H, Lee M, Kim GJ, Hwang JI (2021) The impacts of visual Spine J 28(7):1697–1711 effects on user perception with a virtual human in augmented 9. Zhou Y, Luo H, Yang Y (2017) Implementation of augmented reality conﬂict situations. IEEE Access 9:35300–35312 reality for segment displacement inspection during tunneling 23. Koutitas G, Vyas S, Vyas C, Jadon SS, Koutsopoulos I (2020) construction. Autom Constr 82:112–121 Practical methods for efﬁcient resource utilization in augmented 10. Marto A, Melo M, Gonc¸alves A, Bessa M (2020) Multisensory reality services. IEEE Access 8:220263–220273 augmented reality in cultural heritage: impact of different stimuli on presence, enjoyment, knowledge and value of the experience. Publisher’s Note Springer Nature remains neutral with regard to IEEE Access 8:193744–193756 jurisdictional claims in published maps and institutional afﬁliations.

Journal

Augmented Human Research – Springer Journals

Published: Dec 1, 2022

Keywords: Augmented reality; Food waste; Catering industry; Consumer

References

Environmental impacts of food waste in Europe

Scherhaufer, S; Moates, G; Hartikainen, H; Waldron, K; Obersteiner, G
A simple awareness campaign to promote food waste reduction in a University canteen

Pinto, RS; Dos Santos Pinto, RM; Silva Melo, FF; Campos, SS; Santos Pinto, CMDS
Augmented reality marketing: How mobile AR-apps can improve brands through inspiration

Rauschnabel, PA; Felix, R; Hinsch, C
Representing climate change futures: a critique on the use of images for visual communication

Nicholson-Cole, SA
Implementation of augmented reality support in spine surgery

Carl, B; Bopp, M; Saß, B; Voellger, B; Nimsky, C
Implementation of augmented reality for segment displacement inspection during tunneling construction

Zhou, Y; Luo, H; Yang, Y
Multisensory augmented reality in cultural heritage: impact of different stimuli on presence, enjoyment, knowledge and value of the experience

Marto, A; Melo, M; Gonçalves, A; Bessa, M
Mobile five senses augmented reality system: technology acceptance study

Rodrigues, JM; Ramos, CM; Pereira, JA; Sardo, JD; Cardoso, PJ
Supporting food choices in the Internet of People: Automatic detection of diet-related activities and display of real-time interventions via mixed reality headsets

Fuchs, K; Haldimann, M; Grundmann, T; Fleisch, E
Rendering optimization for mobile web 3D based on animation data separation and on-demand loading

Li, L; Qiao, X; Lu, Q; Ren, P; Lin, R
Projection mapping for enhancing the perceived deliciousness of food

Fujimoto, Y
A framework for extended reality system development in manufacturing

Gong, L; Fast-Berglund, Å; Johansson, B
Web AR: A promising future for mobile augmented reality—State of the art, challenges, and insights

Qiao, X; Ren, P; Dustdar, S; Liu, L; Ma, H; Chen, J
Applicability of immersive analytics in mixed reality: Usability study

Hoppenstedt, B; Probst, T; Reichert, M
Location-based augmented reality with pervasive smartphone sensors: Inside and beyond Pokemon Go!

Shea, R; Fu, D; Sun, A; Cai, C; Ma, X
Support for augmented reality simulation systems: The effects of scaffolding on learning outcomes and behavior patterns

Ibáñez, MB; Di-Serio, A; Villarán-Molina, D; Delgado-Kloos, C
The impacts of visual effects on user perception with a virtual human in augmented reality conflict situations

Kim, H; Lee, M; Kim, GJ; Hwang, JI
Practical methods for efficient resource utilization in augmented reality services

Koutitas, G; Vyas, S; Vyas, C; Jadon, SS; Koutsopoulos, I

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Harnessing Augmented Reality for Increasing the Awareness of Food Waste Amongst Dutch Consumers

Harnessing Augmented Reality for Increasing the Awareness of Food Waste Amongst Dutch Consumers

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Harnessing Augmented Reality for Increasing the Awareness of Food Waste Amongst Dutch Consumers

Harnessing Augmented Reality for Increasing the Awareness of Food Waste Amongst Dutch Consumers

Abstract

Journal

Recommended Articles

References

Our policy towards the use of cookies