Dear all,
The Regulations Challenge aims to push the boundaries of LLMs in understanding,
interpreting, and applying regulatory knowledge in the finance industry. In
this challenge, participants will participate in 9 tasks to explore key issues,
including, but not limited to, regulatory complexity, ethical considerations,
domain-specific terminology, industry standards, and interpretability. We
welcome students, researchers, and practitioners who are passionate about
finance and LLMs. We encourage participants to develop solutions that advance
the capabilities of LLMs in addressing the challenges of financial regulations
and industry standards.
These tasks assess the LLM's ability to handle different types of questions
within the regulatory domain which follow:
- Abbreviation Recognition Task:
Goal: Match an abbreviation with its expanded form.
Input Template: "Expand the following acronym into its full form: {acronym}.
Answer:"
- Definition Recognition Task:
Goal: Correctly define a regulatory term or phrase.
Input Template: "Define the following term: {regulatory term or phrase}.
Answer:"
- Named Entity Recognition (NER) Task:
Goal: Ensure the output correctly identifies entities and places them into
groups that the user specifies.
Input Template: "Given the following text, only list the following for each:
specific Organizations, Legislations, Dates, Monetary Values, and Statistics:
{input text}."
- Question Answering Task:
Goal: Ensure the output matches the correct answer to a detailed question about
regulatory practices or laws.
Input Template: "Provide a concise answer to the following question: {detailed
question}? Answer:"
- Link Retrieval Task:
Goal: Ensure the link output matches the actual law.
Input Template: "Provide a link for ____ law, Write in the format of ("{Law}:
{Link}" or "{Law}: Not able to find a link for the law")"
- Certificate Question Task:
Goal: Select the correct answer choice to a question that may be based on
additional context.
Input Template: "(This context is used for the question that follows:
{context}). Please answer the following question with only the letter and
associated description of the correct answer choice: {question and answer
choices}. Answer:"
- XBRL Analytics Task:
Goal: Ensure the output strictly matches the correct answer to a detailed
question about financial data extraction and application tasks via XBRL
filings. These standardized digital documents contain detailed financial
information.
Input Template: "Provide the exact answer to the following question: {detailed
question}? Answer:"
- Common Domain Model (CDM) Task:
Goal: Deliver precise responses to questions about the Fintech Open Source
Foundation's (FINOS) Common Domain Model (CDM).
Input Template: "Provide a concise answer to the following question related to
Financial Industry Operating Network's (FINO) Common Domain Model (CDM):
{detailed question}? Answer:"
- Model Openness Framework (MOF) Licenses Task:
Goal: Deliver precise responses to questions concerning the requirement of
license under the Model Openness Framework.
Input Template: "Provide a concise answer to the following question about MOF's
licensing requirements: {detailed question}? Answer:"
The final score is determined by the weighted average of metrics for 9 tasks.
We assign the weight of 10% to Task 1-5 each, 20% to Task 6, and 10% to Task
7-8 each.
Important Dates
Training Set Release: September 15, 2024
Training Data Details: Summary of Question Dataset
Validation Set Release: October 30, 2024
Systems Submission: November 7, 2024
Release of Results: November 12, 2024
Paper Submission Deadline: November 25, 2024
Notification of Acceptance: December 5, 2024
Camera-ready Paper Deadline: December 13, 2024
Workshop Date: January 19-20, 2025
Task Organizers
Keyi Wang, Columbia University, Northwestern University
Lihang (Charlie) Shen, Columbia University
Haoqiang Kang, Columbia University
Xingjian Zhao, Rensselaer Polytechnic Institute
Namir Xia, Rensselaer Polytechnic Institute
Christopher Poon, Rensselaer Polytechnic Institute
Jaisal Patel, Rensselaer Polytechnic Institute
Andy Zhu, Rensselaer Polytechnic Institute
Shengyuan Lin, Rensselaer Polytechnic Institute
Daniel Kim, Rensselaer Polytechnic Institute
Jaswanth Duddu, Rensselaer Polytechnic Institute
Matthew Tavares, Rensselaer Polytechnic Institute
Shanshan Yang, Stevens Institute of Technology
Sai Gonigeni, Stevens Institute of Technology
Kayli Gregory, Stevens Institute of Technology
Katie Ng, Stevens Institute of Technology
Andrew Thomas, Stevens Institute of Technology
Dong Li, FinAI
Supervisors
Yanglet Xiao-Yang Liu, Rensselaer Polytechnic Institute, Columbia University
Steve Yang, School of Business at Stevens Institute of Technology
Kairong Xiao, Roger F. Murray Associate Professor of Business at Columbia
Business School
Matt White, Executive Director, PyTorch Foundation. GM of AI, Linux Foundation
Cailean Osborne, University of Oxford
Wes Turner, Rensselaer Center for Open Source (RCOS), Rensselaer Polytechnic
Institute
Neha Keshan, Rensselaer Polytechnic Institute
Luca Borella, PM of AI Strategic Initiative, FINOS Ambassador, Linux Foundation
Karl Moll, Technical Project Advocate, FINOS, Linux Foundation
For more details, please visit https://coling2025regulations.thefin.ai/ or
contact [email protected]
Best regards,
Jimin Huang
The Fin AI (https://thefin.ai)
_______________________________________________
Corpora mailing list -- [email protected]
https://list.elra.info/mailman3/postorius/lists/corpora.list.elra.info/
To unsubscribe send an email to [email protected]
