Skip to main content
universidade lusófona
Outcomes

Protocol

 

Initial Considerations

A well-defined protocol for developing motion recognition software in dance VR experiences is essential to ensure the software's accuracy, consistency, interoperability, security, and quality. It also facilitates collaboration among multidisciplinary teams, promotes continuous improvement, and ensures an immersive and realistic experience for users.

Developing software for motion recognition in VR dance experiences is a complex task. This updated protocol provides a framework for developing motion recognition software in VR dance, specifically adapted to a Unity3D system context, with PICO 4 VR headsets and Awinda XSens motion capture system, considering the specific limitations and challenges of the stage and rehearsal studio environments. It covers everything from initial calibration to ongoing maintenance, ensuring an immersive and accurate experience, with measures to mitigate issues related to visibility, interferences, and movement restrictions. Additionally, it includes a comprehensive safety protocol to protect the health and well-being of the dancers, preventing nausea and ensuring physical safety when using VR technology.

Protocol

1. Definition of Objectives and Requirements

Main Objective: Develop a system that captures and interprets dance movements using the Awinda XSens and reproduces them in real-time in a VR environment with PICO 4 headsets, applicable in both stage and rehearsal studio settings, considering the limitations of the devices and environment.

Functional Requirements:

  • Full-body tracking.
  • Recognition of specific gestures.
  • Real-time VR rendering.
  • Adaptable configuration of the virtual space to match the real stage.

Non-Functional Requirements:

  • Minimal latency.
  • High capture accuracy.
  • Security and privacy of motion data.
  • Solutions for VR headset limitations in environments with poor wall visibility.

2. Technology Selection

  • Tracking Sensors and Devices: Awinda XSens.
  • Development Platform: Unity3D.
  • VR Headsets: PICO 4.
  • Programming Languages: C#.

3. Initial Calibration and Configuration

Calibration Procedures:

  • Follow the Awinda XSens manuals for initial calibration.
  • Perform calibration in both studio and stage environments to ensure accuracy.
  • Configure lights to adequately illuminate the stage walls for the PICO 4 VR headsets.

Environment Configuration:

  • Define the capture space in the studio and stage, ensuring adequate visibility of the walls for the VR headsets.
  • Adjust lighting to avoid interference with sensors and laser video projectors.
  • Restrict movements that require the head to be down or quick movements that might confuse the sensors.

4. Motion Data Capture and Processing

Motion Capture:

  • Use XSens MVN software to capture motion data.
  • Integrate MVN with Unity3D to import data in real-time.

Data Filtering and Cleaning:

  • Implement filtering algorithms in C# to remove noise from motion data.

Pattern Recognition:

  • Use machine deep learning algorithms (such as Long Short-Term Memory (LSTM)) to recognize specific dance gestures.

5. Data Synchronization and Integration

Device Synchronization:

  • Ensure precise synchronization between Awinda XSens and PICO 4 headsets.

Data Fusion:

  • Integrate data from Awinda XSens with the VR environment in Unity3D.

6. Modeling and Rendering

Character Modeling:

  • Create 3D models of the characters in modeling software (e.g., Blender) and import them into Unity3D.

Animation:

  • Map motion data from Awinda XSens to 3D models in Unity3D.

Real-Time Rendering:

  • Configure the virtual stage space in Unity3D to match the real stage, preventing dancers from falling.

7. Testing and Validation

Unit and Integration Testing:

  • Develop unit tests for each component (e.g., data filters, gesture recognition).
  • Perform integration tests to ensure all components work together seamlessly.

Usability Testing:

  • Conduct testing sessions with dancers to evaluate usability and immersion.
  • Adjust allowed movements to prevent VR headsets from turning off.

Results Validation:

  • Compare captured movements with real movements to validate accuracy.

8. Security and Privacy

Data Protection:

  • Implement encryption for storing and transmitting motion data.

Regulatory Compliance:

  • Ensure the software complies with applicable data protection laws.

9. Documentation and Training

Technical Documentation:

  • Create detailed documentation on the integration of Awinda XSens with Unity3D and PICO 4.
  • Include solutions for specific problems encountered (visibility of walls, restricted movements, etc.).

User Guides:

  • Develop user guides for dancers and stage technicians.

Training:

  • Offer training sessions for new developers and end-users.

10. Maintenance and Continuous Improvement

Monitoring and Feedback:

  • Implement tools to monitor system performance and collect user feedback.

Updates and Improvements:

  • Plan regular updates to fix bugs and add new features.

Research and Development:

  • Continue research to incorporate technological advances and improve the VR dance experience.

11. Dancer Safety Protocol

Prevention of Nausea and Discomfort:

  • Acclimation Sessions: Introduce dancers to VR with short sessions, gradually increasing usage time.
  • Regular Breaks: Schedule frequent breaks to prevent eye strain and nausea.
  • Smooth Movements: Prioritize smooth movements and avoid quick transitions that may cause discomfort.
  • Safety Warnings: Inform dancers about motion sickness symptoms and the importance of stopping immediately if discomfort occurs.

Safe Physical Space Configuration:

  • Unobstructed Space: Ensure the capture space is free of obstacles.
  • Clear Delimitation: Clearly mark the boundaries of the physical space corresponding to the virtual space to prevent collisions.
  • Non-Slip Surfaces: Ensure the studio and stage floors are non-slip to prevent falls.

Training and Education:

  • Safety Training: Train dancers in safe VR practices, including how to put on and remove VR headsets safely.
  • Scenario Simulations: Conduct scenario simulations to train dancers in safe movements and how to handle disorientation.

Medical Support:

  • Available Support: Have medical or trained personnel available during sessions for immediate assistance if needed.
  • Symptom Record: Maintain a record of symptoms reported by dancers to identify patterns and adjust the VR experience as necessary.