AI Startup Selection Tools (2024)

Project Overview:

Developed a comprehensive set of technology tools integrating human and AI assessments for selecting climate tech startups in ClimaTech's Great Global Innovation Challenge competition. This project aimed to enhance the accuracy and efficiency of the selection process by leveraging AI for initial screenings and human expertise for final evaluations.

Problem:

Selecting high-potential climate tech startups is challenging due to the subjective nature of evaluations and the large volume of applications. Traditional methods can be time-consuming and may not fully capture the nuances of each startup's potential. There was a need for a streamlined, objective process that combined human judgment with advanced AI capabilities, and reduced bias in selection.

Methodology:

Overview:

As part of Climatech’s Great Global Innovation Challenge, we leveraged advanced artificial intelligence (AI) technology to assess and score each startup on several key dimensions. This process supported a fair, consistent, and comprehensive evaluation of all applicants. The AI utilized natural language processing (NLP) and machine learning models, including OpenAI's GPT-4o, to analyze startup applications based on criteria such as Potential Climate Impact, Solution Creativity, Idea Feasibility, and Team Composition. The scores generated by the AI were used alongside human evaluations to select the startups to present at the challenge.

Background:

Research has demonstrated that "Data-driven initiatives have been shown to help VC firms reduce gender bias and make better, fairer investment decisions" (Data-Driven VC / Source). Additionally, "Machine learning models have already been proven to outperform human investors in deal screening" (Data-Driven VC / Report), helping to identify high-potential startups more effectively. By incorporating AI into our evaluation process, we aimed to ensure a more equitable and objective selection of startups.

Initial Review:

• 57 applications reviewed by an AI tool.
• AI scored applications based on a specific rubric, focusing on Potential Climate Impact, Solution Creativity, Idea Feasibility, and Team Composition.
• Top 36 startups selected for human review based on AI scores.

Semi-Finals:

• Human judges, unaware of AI scores, evaluated startups on team quality, market potential, and technological innovation.
• Composite scores calculated with 75% weighting for human scores and 25% for AI scores.
• Normalized scores using pseudo Z-scores to account for variability among reviewers.

Finals:

• Adjusted weighting to 83.33% for human scores and 16.67% for AI scores.
• AI judges provided real-time questions during pitches, scored presentations, and participated in deliberation.

Key Findings:

• Moderate positive correlation between human and AI scores (Pearson: 0.48, Spearman: 0.43), indicating general alignment with some discrepancies.
• AI provided valuable additional insights, particularly in identifying potential overlooked by human evaluators.
• Continuous refinement of AI models is necessary to improve alignment with human judgment.

User Stories:

• Climate Startup Founder: Have an unbiased selection process for a startup competition.
• Startup competition team, judges, sponsors and investors: Obtain reliable, AI-driven evaluations of startups for informed investment decisions in the climate tech space.
• Mentor/Educator: Access tools that provide clear, data-driven guidance to improve business strategies and pitches for startups.

Acceptance Criteria:

• Startup selection tool: Comprehensive scores based on various criteria such as potential climate impact, feasibility, and team composition.
• AI judge question asker: AI avatar asks an insightful question after startup pitch.
• AI judge scorer: Provides scores across judging criteria dimensions based on startup pitch transcript.
• AI judge deliberator: Participates in judging deliberation to assist in selection of final winners.

Additional Notes:

• Led the project from ideation to deployment, developed website, designed and developed AI tools, and supervised the launch.
• Successfully secured partnership with MIT Climate and Energy Prize and engaged with key stakeholders to ensure alignment with real-world needs.
• The project received attention within the academic and entrepreneurial communities.

Case Study:

• AI tools for startup competitions - A suite of AI-driven tools designed to support judges of startup competitions in identifying high-impact opportunities.
• Actions:
• Conducted market research to understand the needs of startup competition team members.
• Collaborated with an MIT professor to conceptualize and develop AI tools.
• Secured resources and support from MIT.
• Engaged with entrepreneurs and investors to refine tools based on feedback.
• Managed development, stakeholder engagement, and launch activities.
• Results:
• Tools integrated into inaugural Great Global Innovation Challenege
• Positive feedback from users for providing clear, actionable insights.
• Ongoing refinement to meet evolving needs of startup compeition team.

This project exemplifies the successful application of AI to address complex challenges in the climate tech sector, providing valuable support to select startups aiming to make a significant environmental impact.