Automatic Video Labeling and it’s Potential Uses

Automatic video labeling is present today and can be accomplished through several sources. Examples of these are YoloV8, Dataloop AI, and teachable machines. These different video labeling software work by training a Large Language Model (LLM) to recognize specific photos and label objects or subjects as they see fit. When LLMs are trained through several hundreds or even thousands of images, the accuracy of their readings can become very close to, if not 100%, accurate at times. The question is how can we use video labeling to help everyday people, especially the elderly.

The main issue that disabled or elderly people face when they fall is not the impact itself but their inability to call for help. Today, devices made with physical buttons worn around the neck of people susceptible to falls by providers such as Medical Guardian, ADT, LifeStation, etc., can cause a problem when the person becomes incapacitated due to the fall they experienced.

Figure 1: Example of a photo recognition software known as a “Teachable Machine” that can be trained with several hundreds or thousands of photos.

According to the CDC Injury and Prevention Control site, “About 36 million falls are reported among older adults each year—resulting in more than 32,000 deaths” (Moreland, 2020). As for non-fatal falls in adults at or above the age of 65, the amount of falls recorded in 2014 was 29 million, resulting in 7 million injuries. However, because of the development of technology in fall detection, such as the emergency button worn around the neck, the number of falls and fall-related injuries among adults at or above 65 decreased from 2016 to 2018. “Smart technologies are necessary for ambient assisted living (AAL) to help family members, caregivers, and health-care professionals in providing care for elderly people independently” (Zin, 2021). Older adults are at a higher risk of being injured when they fall. Because of this, there is a large number of elderly individuals who do not get to a hospital in time and inevitably pass away.

“As has already been suggested, evaluating fall detection devices needs to become more standardized to properly evaluate the strengths and weaknesses of the currently available devices” (Chaudhuri, 2015). There is no set standard test for testing these models yet. The application of such a concept needs more time for development. With the ever-growing field of AI and LLM training, the future in which lives can be saved because of the technological advancements we have made possible.

Figure 2: visual representation of subject enacting scenario of being off balance and falling to the floor

With this idea of being able to label when an elderly person experiences a fall and gets injured, future applications of these models with higher accuracy in recognizing are limitless. Beyond such benefits for elderly individuals, these models can be placed in nursing homes or even in public settings like stairs. As an example, in the case of an incident, ambulances will respond to the scene faster than someone who can call 911.

An idea that could be brought to light is teaching a LLM to recognize signs of distress or fall/injury in elderly people. The model can be trained by reenacting scenarios where subjects are off balance or have fallen on the floor. This thought process is apparent in a paper by authors Aniqua Nusrat Zereen et al. 2023. Their system consisted of a cloud-based storage, which was responsible for analyzing the detection of falls in their scenarios. “It achieved an accuracy of 91.6% for ordinary actions and fell with a recall of 97.02% for unstable motion” (Zereen, 2023). Through several scenarios, they reenacted, some of which were having a test subject lose their balance, falling, bending over, etc. The model could recognize injury and falls with a 91.6% accuracy.

References

1. “Keep on Your Feet.” Centers for Disease Control and Prevention, 24 Mar. 2023, www.cdc.gov/injury/features/older-adult-falls/index.html.

2. Zereen, Aniqua Nusrat, et al. “Video Analytic System for Activity Profiling, Fall Detection, and Unstable Motion Detection.” Multimedia Tools and Applications, vol. 82, no. 27, Apr. 2023, pp. 42395–415. https://doi.org/10.1007/s11042-023-14993-y.

3. Moreland, B., Kakara, R., & Henry, A. (2020). Trends in nonfatal falls and Fall-Related injuries among adults aged ≥65 years — United States, 2012–2018. Morbidity and Mortality Weekly Report, 69(27), 875–881. https://doi.org/10.15585/mmwr.mm6927a5

4. Zin, Thi Thi, et al. “Real-Time Action Recognition System for Elderly People Using Stereo Depth Camera.” Sensors, vol. 21, no. 17, Sept. 2021, p. 5895. https://doi.org/10.3390/s21175895.

5. Chaudhuri, Shomir, et al. “Fall Detection Devices and Their Use With Older Adults.” Journal of Geriatric Physical Therapy, vol. 37, no. 4, Oct. 2014, pp. 178–96. https://doi.org/10.1519/jpt.0b013e3182abe779.