AI Video Search Engine (AVSE) indexes video content using transcriptions and metadata. It addresses the challenge of finding specific information within the vast amount of video content available online, unlike traditional search engines that primarily index text. The system uses a combination of JigsawStack's embedding model, Supabase for storage, and Hasura for a GraphQL layer to achieve this.
AVSE utilizes a multimodal embedding model for richer semantic understanding. It leverages a GraphQL layer for efficient data querying and access control. The project demonstrates a scalable architecture suitable for indexing large video datasets by using PostgreSQL's pg_vector extension. The use of a newer embedding model supports multiple languages and larger context windows.
- Video Source Indexing: Extracts transcripts, thumbnails, descriptions, and titles from YouTube videos for indexing.
- Vector Search: Performs cosine similarity search on video embeddings to return relevant video results based on user queries.
- Video Clip Extraction: Identifies timestamped segments within videos to play relevant clips based on search results.
- AI Summarization: Uses JigsawStack's API to generate point-form and text summaries of video content.
- Chat Functionality: Provides a chat interface powered by JigsawStack's Prompt Engine for interactive querying of video content.
- Scalability: Designed to handle a large number of videos using PostgreSQL pg_vector extension and optimization techniques.
- Extensible: Supports the addition of new video sources like TikTok, increasing the scope of searchable content.
The project is actively maintained, with recent commits indicating ongoing development and improvements. Comprehensive documentation is provided to guide users through setup and configuration. The project has a growing community and readily available resources, demonstrating a healthy level of community support and reliability.
This project benefits researchers and developers interested in exploring AI-powered video search solutions. It addresses the need for efficient information retrieval from video content, with applications in education, content analysis, and knowledge discovery. It offers a practical demonstration of how RAG and modern AI tools can be used to overcome the limitations of traditional video search methods.