Project Name:

Data Access Alternatives: Artificial Intelligence Supported Interfaces

Contractor: BrightQuery, Inc.

Lessons Learned

• Back-End Setup: The complex AI back-end setup required in-depth collaboration, but this has now been documented and streamlined for future installations.
• SDR Dataset Knowledge: The experience gained in analyzing and processing SDR data will improve efficiency in handling similar datasets moving forward.
• Agency Engagement: Early and sustained communication with external agencies is essential to avoid potential delays.

  • Statistical vs. Conversational Chatbot Approaches:
    • Statistical report-based approaches offer greater certainty and structure, while conversational chatbots provide a more interactive and engaging experience, allowing users the flexibility to explore specific areas of interest in greater detail. In the latest version, we have combined the strengths of both approaches, delivering a solution that integrates the reliability of statistical designs with the adaptability and engagement of conversational systems.
  • Questions and Answers:
    • Questions and answers play a pivotal role in shaping a comprehensive and well-rounded application. They contribute significantly to both the breadth and depth of topic coverage, ensuring that responses are insightful and contextual, meeting a wide range of user needs effectively. We are collecting Q&A’s from all sources including partners and stakeholders to improve the application.
  • Data Availability and Format
    • Consolidated historical data and revisions are critical for accessibility and usability.
  • AI and ML Challenges
    • Commercial AI tools struggle with statistical and time-series data, particularly revisions.
    • Time must be treated as multidimensional, capturing both the period and the timestamp.
  • Standards and Discoverability
    • Schema.org and Croissant standards enhance data discoverability but require additional depth for analytics.
  •  Knowledge Graph Development
    • Triplification is essential for building knowledge graphs but lacks standardization for entity definitions and time-series data representation.
  • Granularity and Interoperability
    • More granular data enhances interoperability but may be affected by changes in methodology or categorization.
  • Hybrid Approach with Conversational Bots:
    • The combination of structured reports and conversational bots enhances accessibility and engagement, making it easier for users to interact with complex datasets and derive insights dynamically.
  • Knowledge Graph Integration:
    • Incorporating knowledge graphs significantly improves data interconnectivity and retrieval, enabling a more holistic understanding of relationships across datasets and facilitating advanced analytical capabilities.
  • Vector-Based Search Optimization:
    • Implementing vector-based search methods has greatly enhanced the accuracy and efficiency of data extraction, allowing users to retrieve highly relevant information with minimal effort.
  • Maintaining Conversational Context:
    • For AI-driven reporting applications, maintaining context across interactions ensures a seamless and intuitive user experience, reducing redundancy and improving information flow.
  • Merging Retrieved and Generated Information:
    • Effectively combining retrieved and AI-generated information enhances data utility, providing users with comprehensive and context-aware responses that improve decision-making.
  • User-Driven UI Enhancements:
    • Iterative updates based on user feedback have proven critical in refining the UI experience, ensuring it meets the evolving needs of researchers and analysts.
  • Quality Control as an Ongoing Process:
    • Regular QC updates have been essential in maintaining the integrity of structured data transformations, ensuring that information remains accurate, consistent, and reliable for downstream applications.

This quarter has set a strong foundation for future advancements, with a focus on refining data transformation, improving UI/UX, and further leveraging AI-driven methodologies.

Click here to learn more about this project

Disclaimer: America’s DataHub Consortium (ADC), a public-private partnership, implements research opportunities that support the strategic objectives of the National Center for Science and Engineering Statistics (NCSES) within the U.S. National Science Foundation (NSF). These results document research funded through ADC and is being shared to inform interested parties of ongoing activities and to encourage further discussion. Any opinions, findings, conclusions, or recommendations expressed above do not necessarily reflect the views of NCSES or NSF. Please send questions to ncsesweb@nsf.gov.