sauron-cv/ROADMAP.md

Sauron Roadmap

Phase 1: Minimum Viable Product

• Communication protocol
  • Determine how our networking protocol will work
    • Protocol must support sending video at low latency
    • Protocol must be lightweight enough to support at least 480p at 15FPS through an Arduino
    • Protocol must have reasonable handling of packet loss or reordering (if we're using UDP, this is especially important)
  • Determine if compression is viable
    • How expensive is compression on microcontrollers (i.e. Arduino)? Do we get better throughput if we just lower framerates?
  • Account for security features, at least to add in the future (encryption? checksum matching? something to make sure that we don't get man-in-the-middle'd?)
• sauron-client
  • Baseline Linux client written
    • Can read data from most USB webcam devices (whatever the most standard format is, we go with it)
    • Can transmit data to sauron-server using communication protocol
    • Can read response data and print to console
• sauron-server
  • Can accept real-time communication from external sources
    • Can read video streams using communication protocol
    • Can execute scripts according to which endpoint the client hits
    • Can gracefully handle lost connection, latency, etc.
  • Can move data into / out of scripts
    • Interprets video stream into single images that scripts can operate on
    • Packages/serializes return data from scripts
    • Handles script errors gracefully
  • Can return serialized data to client
    • Preferably using same data stream
  • Can manage scripts manually through shell interaction / secure file transfer
    • scripts transferred via rsync/SCP are recognized
    • scripts cloned from git are recognized
      • Nice to have: Changes to remote repository are optionally automatically synchronized (either by periodic checks, or by having CI poke the server into updating)
• sauron-bundle
  • Python
    • Support writing scripts in Python (probably a function in a decorator)
    • Handle converting data into OpenCV-friendly format (NumPy array) automatically
• Build tools
  • Build process for sauron-server is configured
    • Builds third-party libraries for scripts with hardware acceleration support (has to be configured by user)
  • Build process for sauron-client is configured
    • Nice to have: Building deb packages (I think that's what pi uses?) with support for software encoding is automated
    • Can be built with hardware acceleration support
  • Testing
    • All functionality has appropriate unit testing written

Phase 2

• Protocol upgrades
  • TBD
• sauron-client
  • Arduino implementation
    • Can read data from SPI camera
    • Can transmit data using (some common) Wi-Fi module
  • Pi Pico implementation
    • Can read data from SPI camera
    • Can transmit data using (some common) Wi-Fi module
    • Probably written in Rust (idk)
  • Identify other commonly used embedded platforms (for implementation in phase 3)
    • Other camera implementations
    • Other common Wi-Fi modules
• sauron-server
  • Improve performace as needed
  • Add access control support (IP whitelisting? API tokens?)
  • Add support for extending with optional modules
    • Create a REST API for admin/user interaction
      • tandem with access control support
  • Add support for compiling C++ scripts
• Optional Web Service Frontends
  • Develop baseline infrastructure for webapps (low-level JS libraries, API interactions, etc.)
  • sauron-admin
    • Users must authenticate using username/password (in existing access control) to access console
    • Authenticated users can manage scripts
      • Users can upload bundle files, or directories in tarball/zip format
      • Users can add new git repository scripts
      • Users can manually pull changes on git repository scripts
    • Authenticated users can manage server configuration
      • Probably just editing config files in a text editor, or provide at least some config in a GUI
  • sauron-liveview
    • Users can see all connected clients
    • Users can view live video from clients
    • Integrated with sauron-admin (if both are installed, admins can view live feed from admin console)
    • Nice to have: Require authentication (this can be turned on/off by admins)
• sauron-bundle
  • Debug images can be generated and served using liveview frontend
  • C++
    • Support writing scripts in C++
• Build tools
  • sauron-server build automation
    • Hardware acceleration configuration can be (somewhat) automated for CUDA
  • sauron-client build automation
    • packages for Pi support hardware acceleration
    • packages for other distros being built
    • Nice to have: Automated linting / compile checks for embedded platforms
  • Testing
    • All functionality has appropriate unit testing written

Phase 3

• TBD