2024-11-21 AIM TF Meeting Notes

2024-11-21 AIM TF Meeting Notes

Meeting Date & Time

  • Nov 21, 2024 

    • 09:00-10:00 PT / 16:00-17:00 UTC 

Zoom Meeting Links / Recordings

Meetinghttps://zoom.us/j/98931559152?pwd=d0ZwM1JHQ3d5cXRqVTh4NlRHeVJvQT09

Recording:

Transcript:

Attendees:

@Wenjing Chu@Neil Thomson, @Neil Thomson@Eric Drury@Savita Farooqui, @Judith Fleenor



Main Goal of this Meeting

Agenda Items and Notes (including all relevant links)

Time

Agenda Item

Lead

Notes

5 min

  • Start recording

  • Welcome & antitrust notice

  • Introduction of new members

  • Agenda review

  • Assigning minutes taker

Chairs

  • Antitrust Policy Notice: Attendees are reminded to adhere to the meeting agenda and not participate in activities prohibited under antitrust and competition laws.

  • ToIP Policy: Only members of ToIP who have signed the necessary agreements are permitted to participate in this activity beyond an observer role.

  • ToIP TSWG IPR Policy: see TF wiki page. AI & Metaverse Technology Task Force. Please remember that TSWG2 has transitioned to a new IPR policy using OWF license.

15 mins

  • Introduction of new members

  • Any general announcements news, that could be of interest to the TF

  • Last week's action items

All

  •  

    • Discussions on how to promote ToIP in LF projects

    • Presentations by @Judith Fleenor and @Wenjing Chu

    • Trustworthiness of open-source software & Cyber Resilience Act will be required by 2027

    • Workshop:  Open Source Software Stewards and Manufacturers Workshop | LF Events

    • Reputation vs Identity of the Committers & Maintainers 

    • Will be CRA "clearing houses" 

    • Pushback by contributors stating that it's open source and should be anonymous, that's why Reputation is more acceptable.

    • @Wenjing Chu will be presenting "From Security to Trust" in the afternoonLFMS news (LF Member Summit is Nov 19-21 in Napa, CA. 25 years of LF.

  • Holiday calendar:

    • 05 Dec is on

    • 19 Dec (@Wenjing Chu out) is off

    • first meeting in 2025 is off

15 mins

White Paper & C2PA and CAWG, TSP



All 

  • Current version

  • Create a new version from scratch in the new year

  • Publish date: to be discussed

10 mins

@Eric Drury AI Safety Businesses & Tools

All

Eric moderated a panel on Cooperative AI Governance' as part of the Paris Peace Forum, and met with a couple of guys who run AI safety businesses with tools "designed to probe the robustness of LLMs against prompt injections" for example and will discuss these kinds of products / businesses - red teaming, blue teaming,

  • All Tech is Human

  • All about governance

  • Macron wanted global collective functioning governance

  • Company on panel on AI compliance: https://alltechishuman.org/ & https://www.prism-eval.ai/

  • @Steven Milstein Reputation and data provenance are more important than identity. 

  • @Neil Thomson Consumer Reports at IIW may be getting into the business of reputation

  • @Savita Farooqui Accountability in Government space is more important than bot or human

  • @Wenjing Chu Regulations for Accountability 

  • Example: Self-driving cars. Who's accountable? Driver not manufacturer.

  • @Eric Drury: Red team: self-auditing vs Blue Team: external 

  • @Wenjing Chu: Provenance of AI data set

    • Tools like C2PA & Trust Spanning Protocol 

    • Boston group of doctors: One group with ChatGPT trained for doctors, one with ChaptGPT not trained, one no ChatGPT.

      • Result: [ ] @Wenjing Chu please complete 

        • Generic ChatGPT had the greatest success rate >90%

        • Use of LLM by the doctors does not improve diagnosis success rate of ~75%

        • Suggested reasons: doctors are not trained to effectively use LLMs to improve diagnosis, they do not trust LLMs.

      • See reference: https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2825395

10 mins

Discussions of guardrails for AI - the lack of, or the insufficient of





5 mins



Chairs

NEW ACTIONS from the previous meeting postponed to the next meeting.