Minutes

04Sep2024

Attendees: Lovemore Gakava, Nicholas Masel, Steven Haesendonckx, Joel Laxamana, Hanming Tu
Facilitator: Joel Laxamana

Agenda :

  1. Introduce ourselves and what we want to get out of this WG.
  2. Discuss and decide how we want to start this off and pick a template document we want to work with first (e.g. ADRG).
  3. How we want to track our work (e.g. github project board - create issues and self-assign to the work we feel comfortable doing)
  4. Frequency of our catch-ups to get status updates on work.
  5. Discuss FDA quarterly meeting presentation :

Hope this time works for you. If not, please let me know. First action is to create a set of slides to present at FDA quarterly meeting : 16 October at 14:00 BST. I was thinking we can create quarto md slides.

Discussion/Issues/Actions :

  1. KO meeting 04Sep2024.
  2. First draft of slides due 13Sep2024 (or sooner).
  • Intro : List out the current ADRG template we have now and how we want to update the template to include open-source metadata - Lovemore
  • Within the current ADRG from Pinnacle 21 vs an ADRG where Open-source submission was used, list out the differences and note these differecnes as the updates we want to make to the current ADRG template to be Open-Source friendly.
    • e.g., List out all of the tables in this ADRG, here, that we can automate and generate sourcing available metadata
      • 1.2 Study Data Standards and Dictionary Inventory (Can this come from the TS SDTM domain?)
      • 3.1 Core Variables (come from the define.xml specificaions)
      • 4.2 Data Dependencies (may need to some code scraping to read through each ADaM program and identify any other ADaMs being sourced in to create the ADaM)
      • 5.2 Analysis Datasets (read in all of the .xpts in the submission package to extract the metadata for each ADaM to fill in this table)
      • 6.2 Issues Summary (extract report from P21)
      • 7.2 ADaM Programs (read in all of the .xpts in the submission package to extract the list of ADaMs and it’s label for each ADaM to fill in this table)
      • 7.3 Analysis Output Programs (this one may be tricky as there are many study specific Selection Criteria to include)
      • 7.4 Proprietary R Packages (Joel has some code for this, in R, essentially using SessionInfo() and {renv})
      • eCTD structure.
  • With these listed updates we want to make, we can utilize metadata to help fill in this information in the ADRG (Let’s list write out specifications as to where )
  • Challenge the file format delivered to the FDA, not only .pdf, but can we additionally submit .qmd files (or other file formats) as well so that the ADRG can be reproduced on the Agency side to help better click on code from the ADRG to help execute. Note : eCTD specifications say we can submit .qmd for analysis, but can we also submit it as a form of ADRG documentation.
  • Do we want to bring up the possibility of creating an open-source package for this to help automate the use of metadata in the ADRG documentation.
  1. Review and Feedback due 20Sep2024.
  • Things to think about :
    • We want this template to be able to support multiple open-source languages. Currently, R is the go-to in Clinical Trial submission now in the industry for open-source. Do we want to create an ADRG based on R as first go or do we want to add the complexity of having to think about multiple open-source programming languages? It would be good to keep the template as simple and user-friendly as possible. Have a think offline.

Actions:

  1. Update and finalize slides due 27Sep2024.
  2. Practice presentation due TBD (anytime between 4 & 6)
  3. Present 16Oct2024

  • Nicholas Masel to

    • find someone from CDISC as a POC for any questions we may have during this project.

    • add Steven H. from JnJ to this WG - Haesendonckx, Steven [JRDBE]

  • Hanming to

    • find out who from the FDA can get us invited to the next FDA quarterly meeting.

    • who will be the assigned FDA person(s) moving forward to collaborate with us on this project.

  • Joel to create base .qmd slides for FDA meeting.

  • All to contribute to the .qmd slides per the timelines above.

18Sep2024

Attendees: Lovemore Gakava, Nicholas Masel, Steven Haesendonckx, Joel Laxamana, Hanming Tu
Facilitator: Joel Laxamana

Agenda :

  1. Get google accounts from the team members to collaborate on google slides for FDA quarterly meeting presentation : Enhancing Clinical Trials FDA Submission Documentation through the Power of Metadata - OSDocuMetaWG

  2. We should connect with the PHUSE WG who created the ADRG templates initially.

  3. We should connect with FDA, Paul and HyeSoo, YounKyeong to gain feedback on where they can see Open-Source documentation can go for FDA submissions.

Discussion/Issues/Actions :

  1. Ensure everyone has access to collaborate and edit the google slides by end of the week.

  2. We need to draft these slides by the end of next week then send to team to review.

  3. Finalize these slides by the end of the Month.

  4. The slides have assignments on them so we know who is working on which content, but feel free to collaboarate on each others content as needed.

  5. Joel to send email for bullet 3.

  6. Lovemore to send email for bullet 2.

16Oct2024


Attendees: Lovemore Gakava, Nicholas Masel, Steven Haesendonckx, Joel Laxamana
Facilitator: Joel Laxamana

Agenda:

  1. Announcement :

    • Presented at PHUSE/FDA Quarterly meeting today. See ‘Home’ page for presentation slides.

  2. Feedback from team on ADRG Template Updates and Process - PHUSE/FDA Quarterly meeting presentation.

  3. Coordination and Feedback from Key Stakeholders

  4. Future Meetings and Next Steps

Discussion/Issues/Actions :

Feedback on ADRG Template Updates and Process:

  • Feedback from Team Members:

    • Presentation went well.

    • The team is awaiting feedback from a key colleague from Pfizer, Christine Rossin who has a stake in updating the ADRG.

    • The necessity of ensuring the ADRG updates align with the expectations of stakeholders. For our first delivery do we want to include everything we mentioned in our slides or piece-meal our work and deliver in iterative approaches with timelines for each milestone?

External Feedback:

  • Plan to check in with Mike Stackhouse to get his input after he reviews the meeting recording.

  • Need to get insights from an FDA contact (Nick’s mention) if they were on this morning’s PHUSE/FDA Quarterly call.

Coordination and Feedback from Key Stakeholders:

  • David Epstein’s Role:

    • David Epstein, involved with ADRG template work in the past, might provide valuable input and help engage more contributors.

  • Communication with David planned post-meeting to gather and facilitate feedback, potentially via GitHub or alternative platforms for those without GitHub accounts.

Future Communications:

  • Follow-up emails post-meeting to ensure any unanswered questions are collected and addressed.

  • Nick will check for additional FDA members interested in the ADRG updates during the upcoming PHUSE meeting.

Enhancements and Tools:

  • Discussed creating a tool and an R package to bring in metadata effectively.

  • Importance of standardizing the ADRG template for future open-source submissions to drive consistency and efficiency in the industry.

Future Meetings and Next Steps:

  • Internal Coordination:

    • Plan to migrate ADRG quarto markdown file to the PHUSE git area to continue work on this.

    • Testing the reproduction of the ADRG with metadata approaches. See List of other Metadata Opportunities slide in presentation.

  • Timeline and Deliverables:

    • Aim to schedule the next working group kickoff by mid-November.

    • Assignments on updating ADRG to be determined.

    • Emphasis on clear planning and setting deliverable dates.

  • Communication Plans:

    • Regular updates and feedback sessions to ensure all members are aligned.

    • Initiatives to include new members from upcoming meetings and engagements.

Action Items:

  1. Follow-Up with Key Contacts:

    • Nicholas to follow up with Mike Stackhouse as well as others in the PHUSE meeting this coming Friday, 18Oct2024, regarding any further feedback or points to consider before we begin work.

    • Joel to contact David Epstein regarding additional member engagement and feedback collection methods from the audience in the FDA quarterly meeting this morning.

  2. Prepare for Next Meetings:

    • Joel to prepare the ADRG markdown file for migration and further testing.

    • Schedule next working group meeting in November 2025 and ensure clear objectives and tasks are assigned.

13Nov2024


Attendees: Lovemore Gakava, Nicholas Masel, Joel Laxamana, Eric Nantz, Sam Parmar
Facilitator: Joel Laxamana

Agenda:

  1. House cleaning
  2. Updates since FDA quartlery meeting presentation
  3. ADRG discussion and steps to consider moving forward.
  4. Actions

Discussion/Issues/Actions

House Cleaning:

  1. Website updated with quarterly presentations to the FDA.

  2. Added last month’s minutes and actions.

Updates since FDA quarterly meeting:

  • No further feedback from the FDA quarterly meeting yet; efforts to reach out to David will continue.

  • Christine Rossin will be contacted by Nick for assistance with ADRG review, including Sam P. in the correspondence.

ADRG Discussion and review:

  • Discussion with Eric and Sam on using Quarto to generate ADRG and sending it as an HTML file in the eSUB package.

  • Need to support hybrid approaches for ADRG versioning (considering various languages, including SAS).

  • Include Christine Rossin in future meetings to discuss ADRG maintenance.

  • Contact vendors like P21 and CDISC core about the ADRG templates and discuss how the enhanced open-source ADRG will integrate. Christine might provide insights on how P21 sources templates from PHUSE WG.

  • Eric and Sam have joined the team.

Actions:

  • Ask Sam to share FDA guidance on mandatory ADRG sections.

  • Create specifications for sourcing metadata from the eCTD structure to generate the ADRG.

  • Nick to contact Christine for the next meeting on 11 December 2024.

  • Consider moving Pilot 3 adrg.qmd to the PHUSE working project repository.

  • Begin programming functions to create metadata information for the ADRG, with reference to Slide 7 of the provided presentation.

  • Templatize open-source instructions for unpacking and re-executing analysis from the eSUB package in eCTD structure.

27Nov2024

Attendees: Lovemore Gakava, Nicholas Masel, Joel Laxamana, Sam Parmar
Guest(s):Christine Rossin (Guest from PHUSE ADRG WG)
Facilitator: Joel Laxamana

Agenda

  • House Cleaning

  • Updates from actions since last meeting

  • New Topics

  • Actions.

House Cleaning

  • Minutes and actions from the last meeting added (follow-up).

  • Q4 reports submitted by Lovemore (link).

  • New members added to the working group’s G-drive.

    • PHUSE WG does have a dedicated workspace to collaborate and save our Project WG files. Contact Alexandra Pearce (link).

Updates since last meeting

  • FDA guidance on mandatory ADRG sections shared by Sam (details provided by Sam below).

  • Create specifications for sourcing metadata from the eCTD structure to generate the ADRG.

    • Joel drafted a template for specifications on sourcing metadata for the ADRG (link).

  • Nick to contact Christine for the next meeting

    • Christine was able to join today’s meeting.

    • Continued conversations with her will be needed for input on the updated ADRG.

  • Moving Pilot 3 adrg.qmd to the PHUSE working project repository

    • Paused as further discussions with P21, CDISC Core and ADRG PHUSE WG still need to be had to align steps for us to contribute.

  • Programming functions for ADRG metadata creation.

    • Paused as further discussions with P21, CDISC Core and ADRG PHUSE WG still need to be had to align steps for us to contribute.

  • Templatizing open-source instructions for unpacking and re-executing analysis

    • Paused as further discussions with P21, CDISC Core and ADRG PHUSE WG still need to be had to align steps for us to contribute.

New topics brought up with Christine R.

  • Collaboration with Christine Rossin on reviewing the ADRG:

    • Christine shared the current ADRG that is in review (unfortunatley public review has been closed, but we are asking for inclusion in the ADRG review).

    • We then shared our Project WG ‘Mission Statement’ and the RCRSWG Pilot 3 ADRG with Christine to review, where some feedback from her included :

      • Exclude eCTD directory structure.

      • ADRG is an m5 directive, no need to mention m1 folder.

      • P21 pre-populates some ADRG sections, but some require manual sponsor updates.

      • Collaborating with P21 to improve metadata provision for the ADRG.

Actions

  • Identify sections P21 populates and doesn’t populate in the ADRG.

    • Joel to review and plan for automation contribution.

  • Determine how to contribute to automation for new ADRG versions. Needs discussion for supporting hybrid approaches (various languages).

    • Lovemore (and Sam P.) to get in contact with Christine R. We can still invite Christine to future meetings as needed.

      • We need to know what the specific changes we want to see updated in the ADRG for open-source submissions as well as the process for submitting updates.

      • We need to know Christine’s preferred method for collaboration.

      • We want to understand the submission process for approval of the updated template.

      • We also want to know the distribution methods for the approved template for the public to use (e.g. Sponsor, Pinnacle 21, CDISC).

  • Further potential support from other Novo colleagues

    • Lovemore to provide colleague’s contact information for joining the WG.

  • Contact Chris Price to find P21 and CDISC core contacts to see if we can get a meeting with them.

    • Nick to get in touch with Chris P.

  • Complete metadata programming specifications (date to be determined, but work on this can still begin to get our ideas out on ‘paper’).

Thoughts for next steps

  • Decide on moving Pilot 3 adrg.qmd (further discussion needed).

  • Templatize open-source unpacking/re-executing analysis instructions.

  • Contact Alexandra Pearce regarding the PHUSE WG workspace (optional as current way of working may be fine, per Nick M).

11Dec2024

Attendees: Nicholas Masel, Sam Parmar, Steven Haesendonckx, Cervan Girard, Eric Nantz
Guest(s):
Facilitator: Lovemore Gakava

Agenda

  • House Cleaning

  • Updates from actions since last meeting

  • New Topics

  • Actions.

House Cleaning

  • Welcome Cervan Girard

  • Minutes and actions from the last meeting added (follow-up).

Updates since last meeting

    • Lovemore to provide colleague’s contact information for joining the WG.

    • Lovemore (and Sam P.) to get in contact with Christine R. We can still invite Christine to future meetings as needed.

      • We need to know what the specific changes we want to see updated in the ADRG for open-source submissions as well as the process for submitting updates.

      • We need to know Christine’s preferred method for collaboration.

      • We want to understand the submission process for approval of the updated template.

      • We also want to know the distribution methods for the approved template for the public to use (e.g. Sponsor, Pinnacle 21, CDISC).

    • Status: Lovemore has already emailed Christine, and we’re awaiting her response on how to proceed with the ADRG template review and future collaboration.

    • Joel to review and plan for automation contribution.
      • Status - Ongoing. Team is encouraged to review the draft specification by Joel and we can discuss in our next scheduled meeting.

    • Nick’s feedback from communications with Chris Price:

      To avoid endorsing proprietary software, Chris recommends not collaborating with Pinnacle 21. Instead, we should develop an open and flexible solution that doesn’t lock sponsors into a specific vendor. Ultimately, we aim to provide a solution that adheres to CDISC core principles while allowing sponsors the freedom to integrate it seamlessly into their existing workflows.

    • Action: To ensure our approach aligns with industry standards, we’ll invite Sam Hume to explain the role of CDISC core. This will help us integrate CDISC standards into our solution effectively.

Discussion Topics

  • Confirm collaboration method with Christine R.

    • Status: ongoing

  • Nick to give update on PHUSE/FDA Computational Science Symposium (CSS) potential slot to have a breakout session:

    • Nick reached out to the organizers and it was agreed that at the moment the breakout session will focus on the use of TEAL: industry adoption and the collaborative unified ecosystem.

    • Action: Discuss with team to decide whether we need a breakout session and if it’s going to be in the EU or US or both session?

  • Decide on moving Pilot 3 adrg.qmd (further discussion needed).

  • Templatize open-source unpacking/re-executing analysis instructions.

    • Discussions:

      • OS computing environment: The idea of having an OS computing environment was discussed but dismissed as not currently viable due to legal and independence concerns for regulatory authorities.
        • Accumulus Synergy: A similar initiative, Accumulus Synergy, was mentioned (Home - Accumulus Synergy). This platform provides a secure way for life sciences organizations and regulatory authorities to exchange information and collaborate.
        • Scope: It was decided that further exploration of these ideas is outside the current scope and can be revisited after the first deliverable.

    • Action: Start on draft of an open-source unpacking/re-executing analysis instructions template. (when?)

Action Items:

  • PHUSE ADRG Collaboration: Follow-up with Christine. (When: ASAP)

  • CSS Breakout Session:

    • Decide whether a breakout session is needed.

    • If yes, decide whether it will be EU, US, or both.

    • Deadline: Before December 18th meeting.

    • Communicate decision to: Nick.

  • ADRG Development & Review:

    • Create a repository: Use pilot3 or pilot4 ADRG as a starting template.

    • Review metadata programming specifications: Provide feedback on specifications.

    • Identify P21 population in ADRG: Identify sections P21 populates and doesn’t populate.

    • Start drafting a template for open-source unpacking/re-executing analysis instructions.

    • Deadline for all the above: Before December 18th meeting (work can begin immediately).

  • Automation Contribution Strategy:

    • Task: Develop an initial draft of the automation contribution strategy.

    • Deadline: Before December 18th meeting.

Future:

  • To ensure our approach aligns with industry standards, we’ll invite Sam Hume to explain the role of CDISC core. This will help us integrate CDISC standards into our solution effectively.

08Jan2025

Attendees: Nicholas Masel, Sam Parmar, Cervan Girard, Eric Nantz
Guest(s): Christine Rossin (Guest from PHUSE ADRG WG)
Facilitator: Lovemore Gakava

Agenda

  • House Cleaning

  • Updates from actions since last meeting

  • Discussion Topics and actions

  • Next Steps

House Cleaning

  • Minutes and actions from the last meeting added (follow-up).

Updates since last meeting

  • CSS Breakout Session:

    • PHUSE/FDA CSS 2025

      • Decide whether a breakout session is needed.

      • If yes, decide whether it will be EU, US, or both.

      • Deadline: Before December 18th meeting.

    • Communicate decision to: Nick.

      • Nick, Joel, LG and Cervan to obtain approval from parent companies regarding attendance

      • Nick to determine if representation is needed from both:

        • USA

        • European

        • Verify European attendance cap/limitations

          • Note: Current attendance restrictions apply specifically to European participants.

      • when: ASAP

  • PHUSE WG, RCSubWG & FDA : ADRG @ Wed Dec 18, 2024 7:15am - 8am (PST) (hyesoo.cho@fda.hhs.gov) discussion:

    • Feedback/comments?

    • Action:
      • pushing this topic to 22-Jan-2025 for team to discuss further.
      • Upload document to Google drive for team to review.
        • who: LG
        • when: 15-Jan-2025

  • PHUSE ADRG Collaboration: Follow-up with Christine. (When: ASAP)

    • who: Sam

    • when: 15-Jan-2025

    • action: add her to the meeting invite

  • ADRG Development & Review:

    • Create a repository: Use pilot3 or pilot4 ADRG as a starting template.

      • who: LG

      • when:15-Jan-2025

      • action: ensure that members have access to the phuse-org github organization

      • repo name : adrg-quarto

    • Review metadata programming specifications: Provide feedback on specifications.

      • check if everyone has access to Google drive?

        • resolved: everyone has access

      • when should everyone provide feedback?

        • when: TBD - pushed to future meeting to discuss the different sections in detail.

    • Identify P21 population in ADRG: Identify sections P21 populates and doesn’t populate.

      • who: Cervan

      • how: use cdisc pilot study and run it through pinnacle 21 and provide report to team.

      • when: 22-Jan-2025

      • Post meeting update: Joel has made a start - see document, https://docs.google.com/document/d/16Pz4NkVP_yC-FsoykENZTaPcV5gDu3LN/edit?usp=sharing&ouid=103991081896680964420&rtpof=true&sd=true. Cervan to review.

    • Start drafting a template for open-source unpacking/re-executing analysis instructions.

      • who: Steven (we have volunteered you :-)) , LG & Cervan - provide NN example.

      • when: 22-Jan-2025

      • action: consider creating a package template to populate the adrg and this might be part of unpacking/re-executing analysis.

  • Automation Contribution Strategy:

    • Task: Develop an initial draft of the automation contribution strategy.

      • drafts of template? to confirm with Joel

    • when: 15-Jan-2025

  • Invite Sam Hume to explain the role of CDISC core.

    • who: Nick

    • when: Joel and LG to provide timelines to Nick by 15-Jan-2025

Discussion Topics:

Meeting Update - Discussion with Christine

Christine joined the last 20 minutes of the meeting (thanks to Sam’s invitation). During her time, we reviewed Hye Soo’s comments and discussed the following key points:

Key Discussion Points

Regarding Section 7 (Appendix: Instructions to Execute Analysis Programs in R):

  • Christine recommends moving this to a separate supporting document

  • The ADRG template should remain software-neutral

  • We should develop separate completion guidelines for different software (R, SAS, and potential future additions)

Next Steps

We need to include Christine in upcoming meetings to determine who will take ownership of writing the R completion guideline and discuss further the content of the template and the R completion guideline.