Introduction
Nostalgia is a smart eyewear device that enhances a sense of place by enabling users to experience location-based memories in situ. A sense of place is defined as “a combination of characteristics that makes a place special and unique” and “foster a sense of authentic human attachment and belonging” (Wikipedia, 2015). In our digital lifestyles, we are increasingly detached from consciously experiencing the physical environment. Too often we move through our surroundings with our heads down and our eyes glued to our mobile devices. Memories and place attachment are closely linked: our sense of attachment and belonging to a place is often shaped by the memories of our experiences that have occurred there. The “sense of place” writes author Rebecca Solnit, “is the sixth sense, an internal compass and map made by memory and spatial perception together” (Solnit, 1999). We designed Nostalgia to enhance the user’s sense of attachment and belonging to physical places by experiencing personally meaningful, context-aware, location-based memories as they move through the built environment.
Nostalgia’s target user group is city dwellers who regularly interact with the urban environment in their daily lives. We chose to focus on urban residents because they have rich histories of place-based memories that have occurred in a dense area. The urban environment, with its wealth of stimuli, also provided us with a challenge when designing for a reflective, context-aware experience. Nostalgia is targeted to urban residents who are engaged in leisure activities and have the attentional resources and emotional predisposition to devote to experiencing place-based memories.
Nostalgia’s target user group is city dwellers who regularly interact with the urban environment in their daily lives. We chose to focus on urban residents because they have rich histories of place-based memories that have occurred in a dense area. The urban environment, with its wealth of stimuli, also provided us with a challenge when designing for a reflective, context-aware experience. Nostalgia is targeted to urban residents who are engaged in leisure activities and have the attentional resources and emotional predisposition to devote to experiencing place-based memories.
Formative Study
We designed our formative study to explore the following research questions:
Methods
Map Probe
We designed a two part map cultural probe to understand:
Part I
First, we asked participants to think of a city or a neighborhood outside of Ann Arbor where many personally meaningful memories have taken place. Next, we gave the participants a blank sheet of paper and sharpies and asked them to draw a map of the places where their memories occurred. We asked them to draw the place and either draw a picture of the memory or annotate their map with a note describing the memory. We instructed participants not to worry about the geographic accuracy or visual appeal of the map, but rather to focus on the experiential aspects of the memory and the place.
Part II
Second, we asked participants to mark locations on a map of downtown Ann Arbor that they felt were important, meaningful, or significant with a small post-it note and to write down the memory and feeling associated with the place. We encouraged participants to verbally describe the memory while they were marking the map and asked interview questions to elicit more detail.
- What makes people feel “connected” or “attached” to a place?
- What types of places and memories are important to people?
- What stimuli trigger the activity of remembering?
- When and in what context are people in reflective moods?
- What behaviors and attitudes result from the act of remembering?
Methods
Map Probe
We designed a two part map cultural probe to understand:
- The relationship between memory and place
- What makes people feel “connected” or “attached” to a place
- What types of places and memories are important to people
Part I
First, we asked participants to think of a city or a neighborhood outside of Ann Arbor where many personally meaningful memories have taken place. Next, we gave the participants a blank sheet of paper and sharpies and asked them to draw a map of the places where their memories occurred. We asked them to draw the place and either draw a picture of the memory or annotate their map with a note describing the memory. We instructed participants not to worry about the geographic accuracy or visual appeal of the map, but rather to focus on the experiential aspects of the memory and the place.
Part II
Second, we asked participants to mark locations on a map of downtown Ann Arbor that they felt were important, meaningful, or significant with a small post-it note and to write down the memory and feeling associated with the place. We encouraged participants to verbally describe the memory while they were marking the map and asked interview questions to elicit more detail.
Diary Study
We designed a diary study to better understand:
We designed a diary study to better understand:
- The activity flow of remembering including memory triggers, context, content, and responses
- What is the memory about?
- What caused you to remember?
- What were you doing when you remembered?
- What did you feel and/or do as a result of the memory?
Collage Cultural Probe
We designed the collage probe to understand:
We designed the collage probe to understand:
- What a "sense of place" means to people on an emotional level
- How people value a sense of place
Findings: Opportunities and Constraints
Map Cultural Probe
We learned from our map cultural probe that some common themes that make places and memories important to our participants are:
Diary Study
Through our diary study, we identified the following the stimuli that trigger the activity of remembering for our participants:
We also learned that people tend to remember the past when they are engaged in activities that require low cognitive effort and when they are alone. This finding shaped our design solution to incorporate context sensing elements that can assess when people would be open to a location-based memory experience and when it would be a nuisance. We wondered if there was an opportunity for Nostalgia to enhance the natural, solitary remembering process by providing a way to experience memories socially in real time.
The most common responses to a memory were feelings or reflections stemming from the content of the memory. The only immediate behaviors associated with remembering that we identified in our study were sharing the memory with someone or distracting oneself from an unpleasant memory. Since we wanted to avoid stimulating a negative emotional experience, we discovered an opportunity to encourage sharing and social behavior through our design.
Collage Cultural Probe
The collage cultural probe showed that our participants feel "connected" or "attached" to place if the place made them feel:
Concept Descriptions
Three design concepts emerged from our formative study:
Nostalgia Headphones
Nostalgia headphones is a pair of "smart headphones" that creates auditory memory experiences. For this concept, we created a storyboard in which a user walks past the Hill Auditorium building and his headphones start playing a symphony that he heard two years ago in the same location. In this design iteration, we explored location and time (2 years) as a trigger for a memory experience and aesthestic content (beautiful music) as the memory. Nostalgia headphones contain a gaze detection sensor and an accelerometer to sense that the user is in a reflective mood.
Map Cultural Probe
We learned from our map cultural probe that some common themes that make places and memories important to our participants are:
- First experiences
- Achievement
- Favorite leisure activities
- Important people
- Trauma/loss
- Aesthetic importance
Diary Study
Through our diary study, we identified the following the stimuli that trigger the activity of remembering for our participants:
- The senses (sight, sound, and taste)
- Location
- Activity
- Seasons
- A combination of multiple triggers
We also learned that people tend to remember the past when they are engaged in activities that require low cognitive effort and when they are alone. This finding shaped our design solution to incorporate context sensing elements that can assess when people would be open to a location-based memory experience and when it would be a nuisance. We wondered if there was an opportunity for Nostalgia to enhance the natural, solitary remembering process by providing a way to experience memories socially in real time.
The most common responses to a memory were feelings or reflections stemming from the content of the memory. The only immediate behaviors associated with remembering that we identified in our study were sharing the memory with someone or distracting oneself from an unpleasant memory. Since we wanted to avoid stimulating a negative emotional experience, we discovered an opportunity to encourage sharing and social behavior through our design.
Collage Cultural Probe
The collage cultural probe showed that our participants feel "connected" or "attached" to place if the place made them feel:
- Safe
- Calm
- Happy
- Reflective
- A sense of belonging
Concept Descriptions
Three design concepts emerged from our formative study:
Nostalgia Headphones
Nostalgia headphones is a pair of "smart headphones" that creates auditory memory experiences. For this concept, we created a storyboard in which a user walks past the Hill Auditorium building and his headphones start playing a symphony that he heard two years ago in the same location. In this design iteration, we explored location and time (2 years) as a trigger for a memory experience and aesthestic content (beautiful music) as the memory. Nostalgia headphones contain a gaze detection sensor and an accelerometer to sense that the user is in a reflective mood.
Nostalgia Watch
Nostalgia Watch is a smart watch that projects visual memory experiences. To explore this concept, we created a storyboard illustrating a user running in the Arboretum. When she stops running, her watch vibrates to indicate that there is a memory to experience. The watch then projects a video of her grandfather running in the Arboretum 50 years before while he was also a student at the University of Michigan. In this concept, we wanted to play with different types of memory content (favorite activities and significant people). We also wanted to see what it would be like to experience a memory that was not one's own.
Nostalgia Watch is a smart watch that projects visual memory experiences. To explore this concept, we created a storyboard illustrating a user running in the Arboretum. When she stops running, her watch vibrates to indicate that there is a memory to experience. The watch then projects a video of her grandfather running in the Arboretum 50 years before while he was also a student at the University of Michigan. In this concept, we wanted to play with different types of memory content (favorite activities and significant people). We also wanted to see what it would be like to experience a memory that was not one's own.
Nostalgia Cardboard
The third design iteration to emerge from our formative study is Nostalgia Cardboard, a virtual reality application that allows the user to experience place-based memories and share them with others. To illustrate this concept, our storyboard details a boyfriend showing his girlfriend around his childhood neighborhood for the first time. Using Nostalgia Cardboard, as he gives his girlfriend a tour of his neighborhood, he can share his childhood memories that happened in his favorite places with her. With the Nostalgia Cardboard concept we wanted to explore what a shared, social memory experience would be like.
Opportunities and Constraints
Through the process of brainstorming these 3 concepts, we developed a series of questions to address in our next study:
Through the process of brainstorming these 3 concepts, we developed a series of questions to address in our next study:
- How do these Nostalgia products access the user's memory data?
- How can we design a device that has an always-on capture of experiences to create memory data? What are the potential privacy concerns in this design?
- How would the device gather additional contextual data to ensure that the memory is a pleasant one for the use?
- How can we design a device that has the ability to share an individual's memory data with other people? Would that be a feature that users would desire?
- How would the Nostalgia Watch user feel about her memories being projected if she was in a context where she wasn’t alone? How could our product design allow for private experiences and/or facilitate shared experiences?
- Would it be desirable to have a feature like voice control that allows more user control over the memory experience or should it stay an automatic and system initiated experience?
Experience Prototype
In our formative study and concept iteration we discovered several design opportunities and constraints that might arise from the implementation of our design. To learn more about how users might react to these potential problems and opportunities, we conducted a series of user enactments to better understand how our future users could respond to issues of :
Methods
We constructed 5 different scenarios based on our main themes. We simulated the social contexts and environments in a classroom in North Quad and asked participants to enact loosely scripted scenarios designed to address the concepts and features of our Nostalgia design that we wanted to test. We recruited 5 participants that fit into our target audience through our friend networks. There were 4 female participants and 1 male participant all of whom were in their 20s. We designed “sets” for each scenario featuring props and paired with a backdrop of digital slides projected on the classroom screen. After participants acted out each scenario, we followed up with interview questions to understand their responses to the user enactment.
User Enactment 1: Always on data capture vs. privacy
We designed the first user enactment to explore the following two questions:
- Always on data capture vs. privacy
- System automation vs. user control
- Public vs. personal memories
- Shared vs. private memory experiences
- Memories with negative or "bittersweet" emotional connotations
Methods
We constructed 5 different scenarios based on our main themes. We simulated the social contexts and environments in a classroom in North Quad and asked participants to enact loosely scripted scenarios designed to address the concepts and features of our Nostalgia design that we wanted to test. We recruited 5 participants that fit into our target audience through our friend networks. There were 4 female participants and 1 male participant all of whom were in their 20s. We designed “sets” for each scenario featuring props and paired with a backdrop of digital slides projected on the classroom screen. After participants acted out each scenario, we followed up with interview questions to understand their responses to the user enactment.
User Enactment 1: Always on data capture vs. privacy
We designed the first user enactment to explore the following two questions:
- How do people react to having their experiences recorded and captured all the time?
- How people react to their personal memories being shared by others without their consent?
- Being recorded at all times
- The best friend sharing a private memory involving the participant with an outside party
User Enactment 2: System automation vs. user control
We designed User Enactment 2 to understand:
In Part II of the enactment, we told the participant to imagine that it was a year after she had attended the concert. We projected an image of the exterior of Hill Auditorium on the screen and told the participant to imagine that she was walking past the building on a nice afternoon when she was leisurely returning home from class. We also gave the participant a pair of bluetooth headphones to wear and played music on them using the Spotify app. We told the participant to stop in front of Hill Auditorium for a few moments and look at the building. Then we switched the music to play the first few minutes of Rhapsody in Blue while the participant was looking at the building.
After the enactment, we debriefed by asking the participants how they felt about the experience, specifically:
We designed User Enactment 2 to understand:
- The participant's preference for a fully automated memory experience as opposed to an experience where the user has more control
In Part II of the enactment, we told the participant to imagine that it was a year after she had attended the concert. We projected an image of the exterior of Hill Auditorium on the screen and told the participant to imagine that she was walking past the building on a nice afternoon when she was leisurely returning home from class. We also gave the participant a pair of bluetooth headphones to wear and played music on them using the Spotify app. We told the participant to stop in front of Hill Auditorium for a few moments and look at the building. Then we switched the music to play the first few minutes of Rhapsody in Blue while the participant was looking at the building.
After the enactment, we debriefed by asking the participants how they felt about the experience, specifically:
- How they felt about an auditory memory experience
- How they felt not having control over the music changing to Rhapsody in Blue to simulate a highly automated memory experience
User Enactment 3: Public vs. personal memories
We designed User Enactment 3 explore:
We then asked the participants:
We designed User Enactment 3 explore:
- How comfortable people feel sharing their personal memories with a public audience
We then asked the participants:
- Would they transfer their personal memory to the public memory bench repository? Why our why not?
- How they feel about other people viewing or experiencing their personal, positive memory with their father?
User Enactment 4: Shared vs. private memory experiences
We designed User Enactment 4 to explore:
We then asked the participants:
We designed User Enactment 4 to explore:
- If people would prefer to experience memories privately or would they enjoy experiencing memories communally with others and are there situations where this feature would be undesirable?
- If our final design should include elements like projecting memories that other people can see or if memories should only be visible to individuals on private devices
We then asked the participants:
- How they felt about the scenario
- How they would feel if their private memories were visibly projected to others
User Enactment 5 : Memories with negative connotations and strangers
We designed User Enactment 5 to understand the following questions:
We followed up by asking them how they felt:
We designed User Enactment 5 to understand the following questions:
- What types of memories are meaningful to people?
- How do people feel about experiencing memories with possibly mixed or negative emotional connotations?
- How interested are people in the memory experiences of people they don't know personally?
We followed up by asking them how they felt:
- About the content of each memory experience
- Seeing a happy memory of their ex girlfriend or boyfriend (a bittersweet or negative emotional memory)
- Seeing an image about someone that they've never met (a stranger's memory linked to the Rackham building). Does this enrich their understanding of Rackham as a place where many communal memories are stored?
Findings: Constraints and Opportunities
Always on Data Capture vs. Privacy
Concept Description
Based on the findings from our user enactments, we proposed the concept of Nostalgia glasses: a personal smart headset that allows the user to experience private, meaningful, location-based memories through an augmented reality display and context-aware sensors. Nostalgia glasses features a camera mounted in the front of the glasses to record images and video of memories as they occur in the moment. Users would have the opportunity to turn off the recording, however, to ensure the user's privacy is protected. In the storyboard below illustrating the concept, a University of Michigan student's Nostalgia glasses record all the memories that she experiences during new student orientation and replays them a year later. The device uses GPS, time, an accelerometer, and a skin conductance sensor to determine what memories to show the the student and when would be appropriate to see them. We also included gesture control in this version of the concept to give the users more control over how they experience their memories.
Always on Data Capture vs. Privacy
- Participants indicated that they wanted control over what memories are captured, stored, and shared. They wanted to give their consent and permission in the recording and sharing of memories.
- If a memory was shared with a public audience, participants expressed a desire to be able to edit, crop, revise, and delete the recorded memories.
- While all the users indicated that they liked an element of automation in the music playing in User Enactment 3 because it was unexpected and interesting, some users described the highly automated experience as too "abrupt".
- From this finding, we discovered that while participants like that the system didn't require user input to trigger a memory experience, they want to be notified when the device triggers a memory experience instead of launching directly into the memory.
- Several users also mentioned that they wanted to be able to opt in and out of the memory experience in case they were busy or uninterested.
- Several participants expressed concern over other people being able to view their faces in the public memory and potentially be able to identify them.
- Other users expressed a willingness to share their personal memory with the public since it was a positive memory. User 1 and 5 did mention, however, that the main reason they wanted to transfer the memory to the bench would be so they could revisit the bench at a later date to relive the memory they had shared with their father. This seemed to indicate that the primary motivation behind uploading the memory to the bench was still for a personal reflective purposes, rather than for the public's benefit.
- From these findings, we discovered that users primarily want to experience meaningful memories about people they care about like friends and family and had mixed feelings about their personal memories being shared with a public audience.
- All the participants had a universal preference for having a personal memory experience as opposed to one that was projected and could be viewed by others.
- All the participants reported that they enjoyed seeing the positive memory of receiving their acceptance letter and seeing themselves at orientation. User 2 thought that it would be important to ensure that these memories only occurred at appropriate times, such as one year after orientation, rather than every time one passed the Rackham building since that could become annoying.
- The participants also felt that seeing a positive memory of their ex boyfriend or girlfriend could leave them potentially upset depending on how much time had passed since they had broken up and the circumstances of the break-up. If enough time had passed and they felt sufficiently "over" their ex, the participants thought that they might feel nostalgic viewing a memory of their date.
- Finally, most participants felt that seeing a memory of a stranger from the past was irrelevant to them and that they would prefer to see memories featuring people they care about.
Concept Description
Based on the findings from our user enactments, we proposed the concept of Nostalgia glasses: a personal smart headset that allows the user to experience private, meaningful, location-based memories through an augmented reality display and context-aware sensors. Nostalgia glasses features a camera mounted in the front of the glasses to record images and video of memories as they occur in the moment. Users would have the opportunity to turn off the recording, however, to ensure the user's privacy is protected. In the storyboard below illustrating the concept, a University of Michigan student's Nostalgia glasses record all the memories that she experiences during new student orientation and replays them a year later. The device uses GPS, time, an accelerometer, and a skin conductance sensor to determine what memories to show the the student and when would be appropriate to see them. We also included gesture control in this version of the concept to give the users more control over how they experience their memories.
Final System Concept
Senses and Records Meaningful Moments
As the user experiences meaningful moments, Nostalgia uses it’s powerful embedded sensors to understand if the context and situation is significant to record and save. Nostalgia's sensors determine whether the event being captured is positive and meaningful to the user or not.
Once Nostalgia has identified that a moment is meaningful and positive for the user, it begins to record video and images of the moment. A recording interface appears in the upper right corner of the glasses display so that the user is aware that the camera is recording. The user has the ability to turn the camera off if she does not want the camera to record and on if the Nostalgia recording does not initiate automatically. The user has the option to sync her social media accounts, devices, and calendar with Nostalgia so that the system has more memory content and contextual information to fine-tune the user's memory experience. While all of the user's memories are stored on the Nostalgia device to ensure privacy, the user also has the option to share her memories to the cloud to enable easier distribution.
Senses Reflective State
Nostalgia uses its sensors to determine whether or not the user is in a reflective mood and is possibly interested in experiencing an augmented memory. Nostalgia will only show the user a memory if it senses that the user is in a calm, relaxed state of mind and engaged in leisurely activities like strolling or sitting. If the user is walking very fast and is rushing to get to work on time, for example, Nostalgia will not initiate a memory experience. Nostalgia assesses a user's interest in a particular place by measuring the user's gaze and eye movement. Nostalgia also identifies the user's location to trigger a place-based memory experience.
Reconnect the User with People and Places
Based on the contextual information that Nostalgia gathers, a "memory bubble" appears on the user's display to notify the user that Nostalgia has identified a memory for her to experience. We chose the visual design of a bubble because in many ways, bubbles are similar to memories: both are ephemeral and delicate. We also wanted to connect emotionally with the user by creating a fun, interactive experience.
By using the gesture of tapping with one finger or "popping the bubble" the user has the ability to opt out of the experience if she is not interested in it. If the user wishes to view the memory, she can use the expand gesture to enlarge the memory bubble. The memory is visually displayed as an opaque overlay over the user's current location to reinforce her positive connection with the physical place and to seem as if the user is reliving the memory. By using hand gestures, she is able to "fast forward", "rewind" and "pause" at different parts of the memory. Finally, whenever she wants to end her memory experience, the user can use a one finger tap and the memory bubble will disappear with a "pop" animation. Nostalgia would also enable voice commands triggered by saying "Nostalgia", although we anticipate that they would be less used in a crowded urban context where users would want to avoid looking like they are talking to themselves. The types of memory content displayed will vary by individual user, but Nostalgia will emphasize memories that feature important people in the user's life.
As the user experiences meaningful moments, Nostalgia uses it’s powerful embedded sensors to understand if the context and situation is significant to record and save. Nostalgia's sensors determine whether the event being captured is positive and meaningful to the user or not.
Once Nostalgia has identified that a moment is meaningful and positive for the user, it begins to record video and images of the moment. A recording interface appears in the upper right corner of the glasses display so that the user is aware that the camera is recording. The user has the ability to turn the camera off if she does not want the camera to record and on if the Nostalgia recording does not initiate automatically. The user has the option to sync her social media accounts, devices, and calendar with Nostalgia so that the system has more memory content and contextual information to fine-tune the user's memory experience. While all of the user's memories are stored on the Nostalgia device to ensure privacy, the user also has the option to share her memories to the cloud to enable easier distribution.
Senses Reflective State
Nostalgia uses its sensors to determine whether or not the user is in a reflective mood and is possibly interested in experiencing an augmented memory. Nostalgia will only show the user a memory if it senses that the user is in a calm, relaxed state of mind and engaged in leisurely activities like strolling or sitting. If the user is walking very fast and is rushing to get to work on time, for example, Nostalgia will not initiate a memory experience. Nostalgia assesses a user's interest in a particular place by measuring the user's gaze and eye movement. Nostalgia also identifies the user's location to trigger a place-based memory experience.
Reconnect the User with People and Places
Based on the contextual information that Nostalgia gathers, a "memory bubble" appears on the user's display to notify the user that Nostalgia has identified a memory for her to experience. We chose the visual design of a bubble because in many ways, bubbles are similar to memories: both are ephemeral and delicate. We also wanted to connect emotionally with the user by creating a fun, interactive experience.
By using the gesture of tapping with one finger or "popping the bubble" the user has the ability to opt out of the experience if she is not interested in it. If the user wishes to view the memory, she can use the expand gesture to enlarge the memory bubble. The memory is visually displayed as an opaque overlay over the user's current location to reinforce her positive connection with the physical place and to seem as if the user is reliving the memory. By using hand gestures, she is able to "fast forward", "rewind" and "pause" at different parts of the memory. Finally, whenever she wants to end her memory experience, the user can use a one finger tap and the memory bubble will disappear with a "pop" animation. Nostalgia would also enable voice commands triggered by saying "Nostalgia", although we anticipate that they would be less used in a crowded urban context where users would want to avoid looking like they are talking to themselves. The types of memory content displayed will vary by individual user, but Nostalgia will emphasize memories that feature important people in the user's life.
Although it is not shown in our envisionment video, we would also like to include an optional feedback mechanism at the end of a memory experience. If the memory was inappropriate or fantastic the user could give the Nostalgia glasses negative or positive feedback. Based on this user input, the device fine-tunes its memory selection, triggering mechanism, and context sensing to make sure it is showing the user meaningful memories at appropriate times. At the end of an ideal interaction with Nostalgia, the user feels refreshed by her positive memory experience experiences a reinforced connection with her personal identity, her social identity, and the physical places of her city. Using the Nostalgia platform, the user can reach out to people to share her memory experience by contacting them or sending them the memory.
Envisionment Video
We created an envisionment video to create an experiential prototype of Nostalgia from the user's perspective. We used a video camera, a pair of glasses, After Effects, Final Cut Pro, and Illustrator to implement our video. The video illustrates Nostalgia:
- Sensing significant experiences
- Recording meaningful moments
- Sensing the user's reflective mood, context, and location
- Enabling the user to "relive" positive, place-based memories
- Enabling user control over the experience (we do not show the hand gestures that correspond to the animation of the UI interactions, but imagine that the user is controlling the system with her hands)
- Connecting the user to a place (the book shop) through the memory of a close friend
System Proposal
Product Design
- We designed Nostalgia's frames to have a stylish, lightweight, and discreet form factor so that users would actually feel comfortable wearing it. Many augmented reality glasses make the user look like a cyborg out of a science fiction novel. In our vision of the future, pervasive computing devices will take inspiration from elegant, simple, and timeless forms and materials without compromising their "smart" functions.
- To complete the Nostalgia product design, we also included a concept proposal for the glasses case and packaging. Since Nostalgia is a purely experiential product with little practical benefit, we thought it was important to pay close attention to creating an appealing aesthetic experience. We chose a script typeface for Nostalgia's logotype and a coral accented color palette to emphasize the concept's romantic and sentimental quality.
Technical Features
Camera and Audio
Camera and Audio
- A 4K ultra HD camera mounted in the front of the glasses that records images and videos of meaningful moments that the user will later experience as “memories”. Users can turn off the camera at anytime to ensure their private moments are respected.
- High fidelity audio capture and an ear piece (we did not design this yet) to enable an auditory component to the memory and voice commands.
- A 3-axis accelerometer and 3-axis gyroscope to gain accurate information about the user's activity level.
- A GPS sensor to sense and record the user's location.
- A skin conductance sensor and HRV sensor on the frames near the user's temple to sense the user's affective state.
- Computer vision algorithms and 3D object detection for accurate facial and place recognition and AR implementation.
- Gesture recognition to implement gesture controls.
- A machine learning algorithm that improves the user's memory experience based on user feedback to ensure that the memories are pleasant, meaningful, and relevant.
- Natural language processing capability to implement voice commands.
- The ability to get relevant memory content and contextual data from the user's social media platforms and other computational devices to enhance the accuracy of its memory recommendation.
- Built-in bluetooth and Wi-Fi that enables Nostalgia to sync with other devices, networks, and cloud storage.
- A rechargeable, long-lasting, lithium-ion battery.
- A mobile chip with a powerful processor, "cognitive computing" capabilities (like Qualcomm's and Intel's new wave of chips), and a massive amount of on device memory storage.
- An augmented reality lens display.
Discussion
Technical
Technological advances in battery life, mobile chip processing power, memory storage, computer vision, natural language processing, cameras, and augmented reality will soon easily make a concept like Nostalgia possible. Indeed, many of these features already exist in products like Google Glass. The challenge comes in designing algorithms that can accurately recommend fine-tuned, significant memory experiences. It is a veritable challenge to design an system that will make emotionally meaningful human experiences out of the massive amounts of data that will be constantly captured and stored through devices like Nostalgia. This is partially due to the current lack of affective sensors that can accurately sense a full range of human emotions. We tried to compensate for this by including two types of affective sensors and user feedback in the hope that by triangulating our emotional measurement strategy we would have more accurate results. In order to fully implement Nostalgia, we would also have to more thoroughly investigate the audio and voice command aspects of they system which due to time constraints we did not flesh out fully.
Usability
Nostalgia is largely a conceptual product and when we were designing it we intentionally ignored the fact that very few people will want to constantly wear a pair of glasses that only serve as an augmented memory device. In reality, a concept like Nostalgia would be more far usable as an application for a multi-use smart headset such as Google Glass. Finally, there is a good chance that despite our best efforts our system would still be disruptive and intrusive with "memory bubbles" popping up at inappropriate times and potentially showing unimportant or disturbing memory content. To make sure Nostalgia was truly usable, the context-aware components and algorithms would have to be seamlessly executed.
Social
The most important open questions with Nostalgia are the social issues that could arise from its implementation. Privacy is a major concern when designing a device that can always capture and store massive amounts of memory data. Since we largely focused on the experiential component of the system, we would have to explore the possible ways that people could misuse a product like Nostalgia and ways to prevent them from doing so. In addition, the gesture and voice command aspects of Nostalgia are still not quite socially acceptable and are often seen as embarrassing in a public setting. We would have to investigate ways to make these commands more subtle and natural. Finally, our product design and concept might appeal primarily to a younger, female audience. Since the majority of our research was conducted with young women and we have a young, female design team, we would have to implement additional studies to account for the male perspective and the perspective of people of different age groups.
Technological advances in battery life, mobile chip processing power, memory storage, computer vision, natural language processing, cameras, and augmented reality will soon easily make a concept like Nostalgia possible. Indeed, many of these features already exist in products like Google Glass. The challenge comes in designing algorithms that can accurately recommend fine-tuned, significant memory experiences. It is a veritable challenge to design an system that will make emotionally meaningful human experiences out of the massive amounts of data that will be constantly captured and stored through devices like Nostalgia. This is partially due to the current lack of affective sensors that can accurately sense a full range of human emotions. We tried to compensate for this by including two types of affective sensors and user feedback in the hope that by triangulating our emotional measurement strategy we would have more accurate results. In order to fully implement Nostalgia, we would also have to more thoroughly investigate the audio and voice command aspects of they system which due to time constraints we did not flesh out fully.
Usability
Nostalgia is largely a conceptual product and when we were designing it we intentionally ignored the fact that very few people will want to constantly wear a pair of glasses that only serve as an augmented memory device. In reality, a concept like Nostalgia would be more far usable as an application for a multi-use smart headset such as Google Glass. Finally, there is a good chance that despite our best efforts our system would still be disruptive and intrusive with "memory bubbles" popping up at inappropriate times and potentially showing unimportant or disturbing memory content. To make sure Nostalgia was truly usable, the context-aware components and algorithms would have to be seamlessly executed.
Social
The most important open questions with Nostalgia are the social issues that could arise from its implementation. Privacy is a major concern when designing a device that can always capture and store massive amounts of memory data. Since we largely focused on the experiential component of the system, we would have to explore the possible ways that people could misuse a product like Nostalgia and ways to prevent them from doing so. In addition, the gesture and voice command aspects of Nostalgia are still not quite socially acceptable and are often seen as embarrassing in a public setting. We would have to investigate ways to make these commands more subtle and natural. Finally, our product design and concept might appeal primarily to a younger, female audience. Since the majority of our research was conducted with young women and we have a young, female design team, we would have to implement additional studies to account for the male perspective and the perspective of people of different age groups.
Conclusion
Nostalgia is an augmented reality headset that strives to reconnect urban residents with physical places through reliving personal, meaningful memories that occurred in them. To achieve this goal, Nostalgia:
- Emphasizes the places and memories related to important people, such as friends and family
- Uses a combination of “triggers” (visual, time, and location) to initiate a "natural" augmented memory experience
- Deploys a combination of sensors and algorithms to ensure that 1) the initial moment captured is meaningful to the user and 2) the context is appropriate later to show a memory when users are engaged in leisure activities and are in a reflective mood
- Enables user control over what memories are captured, stored, viewed, and shared and how the memories are experienced.
- Enhances users' connection to place, self, and the people they love
References
Sense of Place. (n.d.). Retrieved December 16, 2015, from https://en.wikipedia.org/wiki/Sense_of_place
Solnit, R. (1999). Savage dreams: A journey into the landscape wars of the American West. Berkeley: University of California Press.
Solnit, R. (1999). Savage dreams: A journey into the landscape wars of the American West. Berkeley: University of California Press.