Evolution of Search, Impact of AI on Product Management
Read all about evolution of technology for Search and how AI is transforming the PM landscape with Rajan
Welcome to the fourth edition of my fortnightly newsletter.
I am sharing a summary of my super interesting prodcast discussion with
Managing Partner, Upekkha where we explored AI's transformative potential, especially with GenAI, and dived into some fascinating topics. Watch the full video hereSearch - One of not so over rated feature but which is the backbone of many products. In a recent engagement that I have started working on, search is critical function of discovery, acquisition, engagement and retention. I started to get more curious about how the paradigm of tech and the user experience has evolved in the last 2 decades.
Product Talk with Malthi
The Evolution of AI
Rajan argues that AI is currently underhyped. He witnessed the early days of AI in 2007, when he felt it wasn't addressing real problems. However, he sees a significant shift with the advent of Generative AI (Gen AI) in 2022, exemplified by tools like ChatGPT. He believes Gen AI, accessible through APIs, will revolutionize industries as profoundly as cloud computing did.
The Global AI Landscape
Rajan acknowledges the uneven distribution of AI technology. While Gen AI hasn't fully arrived yet, India has seen significant investment ($165 million) compared to the US ($30 billion). He highlights India's strength in application-focused AI, contrasting it with the US focus on platform infrastructure.
Identifying AI Opportunities
Rajan emphasizes the importance of identifying genuine problems that AI can solve. He encourages a "build first" approach due to the rapid evolution of AI technology. He cites Alexa as an example where initial use cases were basic but evolved significantly. He emphasizes that AI empowers decision-making and highlights areas where AI can add value, like language translation.
Building and Scaling AI Businesses
Rajan acknowledges the challenge of product-market fit, especially for startups that often pivot multiple times. He proposes a "Crawl, Walk, Run" framework:
Crawl: Focus on the core use case and validate problem-solution fit.
Walk: Address business fundamentals like customer acquisition, retention, and monetization.
Run: Scale the solution, focusing on accuracy, reliability, and user experience.
He emphasizes the importance of exceeding expectations and highlights the evolving role of product managers in the AI era.
The Impact of AI on Jobs
While AI tools like ChatGPT are powerful, Rajan believes product managers won't be replaced. Their core skills of user research, prioritization, and business acumen remain crucial. However, members with computer science or data backgrounds might have an edge in translating AI requirements.
Unconventional AI Applications
Rajan showcases innovative AI applications like:
Suno.ai: Generates professional-quality music in minutes, transforming music creation.
Spine AI: Converts iPhone photos of high-end cars into 3D models in minutes, saving weeks of work.
UnitedWeCare: Uses AI as a "digital twin" to provide therapy to patients, potentially saving professionals 20 hours of meeting time.
Conclusion
Rajan emphasizes that AI has the potential to be a game-changer, offering 0% or 10x growth depending on the use case. He encourages exploration and adaptation to leverage the power of AI in the tech industry and startup world.
YouTube -
Spotify -
Apple Podcast -
The Evolution of Search: From Early Days to AI-Driven Intelligence
The evolution of search technology is a fascinating journey that has transformed how we access and interact with information. From the rudimentary directories of the early internet to the sophisticated, AI-driven search engines of today, each milestone in this journey has significantly impacted our digital experience and enhanced user experience.
Early Days: Directories and Simple Search
In the early 1990s, when the internet was a nascent technology, and finding information was a challenge. The first search engines were essentially directories, with Yahoo! being a prime example. Launched in 1994, Yahoo! started as a manually curated directory of websites. Users navigated through categories and subcategories to find the information they needed. Folks from this era will remember the pain😊
The user experience was quite basic, as users had to manually browse through categorized lists of links. The process was time-consuming and often frustrating, as it depended heavily on the directory's structure and completeness.
By 1996, search engines introduced more automated ways to index and search the web. AltaVista was notable for its full-text search capabilities, allowing users to search for specific words within the text of web pages, which was a significant improvement over directory-based navigation. However, these early search engines often struggled with relevance, as they primarily relied on keyword matching.
The Search Revolution – simple and saviour
In 1998, Google introduced a new algorithm called PageRank, which evaluated the importance of web pages based on the number and quality of links pointing to them. This innovation drastically improved the relevance of search results, propelling Google to become the dominant search engine.
Google's search interface was simple and clean, which significantly enhanced user experience by providing faster and more relevant search results with less clutter. Other innovations like Google AdWords and Google Images in 2001, which enabled users to search for images made search more efficient and user-friendly and visually-richer.
Bing provided rich visual experiences and deep integration with Windows and other Microsoft products like Cortana, enhancing user engagement.
DuckDuckGo, launched in 2008, emerged as a privacy-focused search engine, gaining popularity for its stance on user data protection. It emphasized no tracking, no data storage, and user anonymity, addressing growing privacy concerns among users and enhancing their trust and satisfaction.
WolframAlpha, launched in 2009, took a unique approach by providing a computational search engine capable of complex computations and detailed answers to scientific, mathematical, and technical queries. This specialized search engine used curated and verified data sources to deliver precise answers, enhancing the user experience for niche search needs.
The Rise of Personalization and Context powered by Mobile adoption
Leveraging user data enabled search engines to show more relevant results and created more personalized experience. By looking user search histories, search engines started showing more relevant outputs.
In 2012, Google introduced the Knowledge Graph, a database of billions of facts about people, places, and things. This technology helps search engines understand the relationships between different entities, providing users with richer and more contextual information. For example, searching for a famous person can display a Knowledge Panel with a summary, key facts, and related links, enhancing the search experience beyond simple keyword matching.
Mobile internet usage necessitated context-aware search capabilities. Search engines started to consider factors like user location and device type to provide more relevant results. For example, searching for "restaurants" on a mobile device would yield results for nearby dining options. These advancements made search results more relevant and timely, enhancing the user experience by providing information that was tailored to individual needs and contexts.
Semantic Search and AI Integration
The integration of artificial intelligence marked a significant leap in search technology. In 2013, Google introduced the Hummingbird algorithm, which improved the understanding of search queries by considering the context and intent behind them. This shift towards semantic search meant that search engines could interpret complex queries more accurately, resulting in more relevant search results and a better user experience.
The introduction of AI models like BERT (Bidirectional Encoder Representations from Transformers) in 2019 revolutionized search once again. BERT allowed Google to understand the nuances of language, improving the relevance of search results for complex and conversational queries. This was a game-changer for voice search and virtual assistants, making interactions more natural and intuitive, thereby significantly enhancing user satisfaction.
Let us take an example -
Understanding Your Travel Dreams:
Beyond Destinations: Imagine searching for "romantic getaway with beautiful beaches" instead of just a specific city or island. Semantic search can decipher your intent and recommend destinations known for couples, with stunning coastlines, even if those exact keywords aren't present.
Activities and Interests: Searching for "hiking trails in national parks" would surface vacation packages or rentals near relevant parks, not just generic travel deals to those regions. Semantic search understands you're looking for an active trip focused on nature.
Similarly, Yandex, the most popular search engine in Russia, leveraged advanced language processing to provide excellent understanding of the Russian language and grammar. Its integration with various Yandex services like maps, mail, and news created a comprehensive and engaging user experience.
The Impact of Deep Learning and Neural Networks
Deep learning and neural networks have further advanced search capabilities. Technologies like convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have improved image and video search by enabling search engines to understand and categorize visual content accurately. Transformer models, including BERT and GPT, have greatly enhanced text-based search by capturing the context and semantics of language more effectively.
These advancements have made search engines capable of providing more accurate and contextually relevant results, improving user experience by delivering precisely what users are looking for.
Vector Search and Semantic Understanding
Vector search represents a major leap in search technology. By converting words, phrases, and entire documents into dense vectors, search engines can perform more accurate and context-aware searches. Techniques like Word2Vec, GloVe, and BERT embeddings allow search engines to understand the semantic meaning of queries and content, improving the relevance of search results.
Vector search has enhanced user experience by making search results more accurate and contextually relevant, reducing the effort users need to find the information they seek.
Let us take some examples:
E-commerce:
Fashion: You find a picture of a dress you love online but don't know the brand or style. Vector search finds similar dresses based on cut, color, pattern, and material, even from different stores.
Electronics: Searching for "a good laptop for photo editing" surfaces laptops with powerful processors and high-resolution displays, even if those terms aren't explicitly mentioned in the product descriptions.
Social Media:
Image Recognition: Social media platforms can recommend content based on the content of your photos. If you post a picture of a delicious meal, you might see recommendations for restaurants or recipes.
Personalized Feeds: News feeds can be tailored to your interests by understanding the topics and themes you engage with, even if they aren't explicitly mentioned in the posts.
Baidu, the leading search engine in China, excelled in language optimization and multimedia search capabilities, providing strong services for music, videos, and images. Its localized services, including maps, news, and encyclopedias, made it a comprehensive tool for Chinese users, enhancing their search experience.
Future Directions: AI-Driven Search and Beyond
Looking ahead, the future of search is set to be even more transformative. AI and machine learning will continue to drive innovations in personalization, context-awareness, and relevance. Predictive search capabilities will anticipate user needs, providing information before a query is even made. This proactive approach will further enhance user experience by reducing the time and effort required to find information.
Visual and voice search will become more sophisticated, allowing users to search using images, videos, and natural language. The integration of augmented reality (AR) and virtual reality (VR) will create immersive search experiences, making information more accessible and engaging.
AI will also enhance ethical and responsible search practices. Efforts to reduce bias and improve privacy will be crucial, ensuring that search engines provide fair and secure experiences for all users.
Some examples:
Immersive Search Experiences:
Augmented Reality (AR): Imagine using your phone's camera to virtually place furniture in your living room to see how it looks before you buy. AR could also be used to try on clothes virtually or see how makeup would look on your face.
3D Visualization: Spin and zoom in on product images to get a 360-degree view and a better sense of scale and detail. This would be particularly helpful for furniture, appliances, and other complex products.
Search by Image and Voice:
· Visual Search: Imagine snapping a picture of a friend's stylish jacket and instantly finding similar products for purchase online. No more struggling to describe what you're looking for in text.
· Multimodal Search: Combine voice and text queries for a nuanced search. "Show me comfortable running shoes with good arch support, under $100." The search engine would use your voice query for intent ("comfortable") and text for specific details ("arch support," "price").
Conclusion
As technology continues to advance, the future of search promises to be even more conversational, transforming how we interact with the world’s information and significantly enhancing the user experience. This journey underscores the importance of search as a fundamental tool in our digital lives, continually evolving to meet our ever-changing needs.