TL;DR

iLightOCR is an AI-powered productivity tool designed to help readers save and summarize article snippets while on the go. Users can easily highlight text, screenshot it, and let iLightOCR transform these into concise summaries saved in a personalized Notion database.

Problem Statement

People often lose valuable insights and quotes when reading articles on their phones due to the lack of an efficient tool to save and summarize specific text snippets. Current solutions are cumbersome and mainly focus on saving entire articles or are limited to browser extensions, leaving mobile users frustrated and inefficient in their reading habits.

Solution Overview

iLightOCR provides a comprehensive solution by allowing users to highlight text, take screenshots of articles, and process these images to extract and summarize the content. The summarized text is then saved directly to the user's Notion database, ensuring easy retrieval and organization.

Core Features

• Highlight and save snippets: Users can highlight text and take screenshots of articles directly from their mobile devices.
• Text extraction: Automatically convert highlighted and screenshotted text into a digital format.
• Content summarization: Identify the key points from the extracted text.
• Notion Integration: The summarized content is saved directly to a user's Notion database for easy retrieval and organization.

Key Benefits

• Easy and efficient experience: Streamlines the process of capturing and summarizing important text snippets from mobile devices.
• Integrated: Integrates seamlessly into daily workflows, making information retrieval easy and intuitive.

Demo Tech Stack

• Computer Vision: OpenCV
• OCR: Azure, Tesseract
• AI: ChatGPT
• Databases: AWS, Notion
• UI: Streamlit

Demo Caveats

• Performance varies for non-white or non-black article backgrounds
• No options to toggle between different OCR and LLM providers
• Clunky uploading process; want to build share-sheet functionality
• Other project opportunities: Text classifier to better categorize and tag articles; scrape full article text based on snippets, not just the URL;

What I learned

• How to wrangle different file and image formats across Streamlit, OpenCV, Scikit-Image, and more.
• How to run optical character recognition and leverage image manipulation libraries improve text extraction performance.
• How to clean and structure data so that disparate APIs can work together seamlessly.
• How to integrate multiple APIs and services into a single project, including Azure OCR, Tesseract, and Notion.
• How to leverage OpenAI's APIs to accomplish different subtasks within a single project.
• How to fast-track UI and event handling with Streamlit.

TL;DR

RIKU is an AI chatbot that gives gamers convenient access to helpful information, statistics, maps, and guides about their favorite video games. Users can select a video game franchise, then ask questions to retrieve specific information, allowing gamers to find answers to their in-game queries efficiently.

Problem Statement

Gamers often struggle to find specific information in the vast sea of internet resources while playing. Switching between the game and various online resources such as blogs, wikis, and guides disrupts the gaming experience and is time-consuming. This issue is particularly pronounced in complex game genres like RPGs and MMOs, where players need detailed information to optimize their gameplay strategies.

Solution Overview

RIKU addresses this problem by providing a centralized AI-powered chatbot that pulls information from wikis and other reliable sources. It offers precise and quick answers to user queries (e.g., “where can I find item X?, what’s the hidden ability for class Y?”), thereby enhancing the gaming experience with reliable information at gamers' fingertips.

Core Features

• Accept user questions: Allow users to input questions about specific game details (e.g., what a certain debuff does in Xenoblade).
• Collect game data for key titles: Collect and store information from pre-selected resources (e.g., Fandom wikis) for various games.
• RAG-capable AI: Integrate ChatGPT so that RIKU can answer user queries dynamically, then fine-tune the model on this specific dataset (e.g., question-and-answer format based on parsed Fandom wiki data).

Key Benefits

• Easy and efficient experience: Streamlines the process of finding specific game information, reducing the time gamers spend searching through multiple sources.
• Comprehensive and centralized information: Aggregates and synthesizes information from multiple sources or pages, providing a centralized platform for game-related queries.

Product Flow

• RIKU scrapes and collects game data: Scrape, parse, and organize information for select titles in a way that’s accessible for the AI.
• User asks a question about a supported title: The user inputs a specific question about game details (e.g., "when does Metang evolve in Pokemon Ruby?”).
• RIKU generates an answer:The AI model processes the user's question, matching it with the parsed data from the wiki to find the most relevant information via RAG.
• User continues the conversation or continues on with their gameplay: Follow-up questions and more detailed responses available if needed.

Demo Tech Stack

• AI: ChatGPT
• RAG and Vector Store: LangChain
• UI: Streamlit
• Web Scraping: Wikiextractor Package

Demo Caveats

• Limited to Xenoblade Chornicles 3 Fandom wiki data
• Limited to queries on classes, skills, arts, builds, and gems
• Locally hosted vector store with out-of-the-box embeddings
• Next steps: larger breadth of games and questions, voice-chat

What I learned

• How to organize, chunk, and store data for efficient AI training and inference.
• how to leverage embeddings and vector stores to improve AI chatbot performance via RAG.
• How to scrape public web data.
• How to clean and strcuture web data for AI training and model fine-tuning.