ICAT Project


ICAT Collaboration Meeting - 30th November 2023



  • Alejandra Gonzalez-Beltran
  • Rolf Krahl
  • Salman Matalgah
  • Louise Davies
  • Gabriel Preuß
  • Viktor Bozhin
  • Alan Kyffin
  • Kistry Syder
  • Patrick Austin (late)


  • Chris Prosser
  • ESRF

Site Updates


lunch production ICAT - successful, adapted new front-end - ICAT+ based on their approach, ingestion using some metadata coming from user support software and data acquision group, ingestion with that technique Integrating authentication w/ Active Directory - working perfectly Will be ready soon (1-2 months), some concerns about making it open to the public internet but resolvable

Gave a demo of their ICAT + DataHub stack

Simple, but getting more functionality is harder Download working ESRF offered the HDF5 browser, working at SESAME

Alejandra: is your data on tape? Salman: long term storage under construction, short term storage is 0.5PB (accessible by both NFS/SAMBA and data portal)

  • going to work with ESRF on backup solution

Open data access & data policy is WIP as well

Salman: keen to write a paper to document our knowledge - welcome for suggestions for conferences

  • Rolf suggested NOBUGS if they don't mind waiting


not much to report

ISIS: (update from Louise)

Fixing some metadata (facility cycles) New starter working on ICAT stack

DLS: (update from Kevin)

Nothing that affects collaboration

Component Updates


Payara 6 release on the way (2.0.0)

  • thought to update ids.plugin 2.0.0
  • no changes just want to build for Java 11

Miredot documentation still not building Alan: I raised issue to Miredot, still haven't fixed it Rolf: did you upload the docs manually? Alan: yes Rolf: probably should do the same then for ids

Alejandra: noticed some work from Gabriel being done? Rolf: that's a future release - part of the refactoring work

  • will take a long time
  • will be 3.0.0 release
  • internal redesign, not planned to change the behaviour
  • after this, various open issues can be considered
  • hopefull can streamline various forks into one again

Gabriel added automatic linting recently

  • first step to looking at the whole codebase

Alejandra: plan to change things e.g. that it's single-threaded? Rolf: core is single threaded, but e.g. web handlers and deferred handlers (?) are multi-threaded Synchronised locking holds back perfomance - this could be tackled once codebase is cleaner


1.2.0 release Minor change

  • re: moving HZB ingest internal code into python-icat
  • others mentioned they were interested at the F2F meeting
  • fix linking between samples and datasets(?)

Kevin: is the documentation Rolf: yes, but it needs some improvement! It reads XML ingest file (ingest into ICAT, not ingest data) It's restricted to only what's possible to ingest Kevin: could be used for CLF who needs a new ingestion system Rolf: I'm happy to help, XML file document format isn't well documented Kevin: I'll come and speak about it Rolf: there are some example files in the release


Created an issue on repo Updated config file, ansible setup now fails Could someone fix it? Alan: I'll take a look today

icat.oaipmh, authn.oidc

Release for Payara 6 Dependabot alert for logback - will need another patch release


No updates from STFC components


Next Collaboration Meeting / meeting schedules

change of time - earlier in the day so that Jordan colleagues can join more easily, potentially different day as Thursday is final day of week for Jordan colleagues as well

Maybe move to 1pm GMT?

  • it's still a Thursday for Jordinian colleagues
  • Patrick: can't go to early, need to consider Brazil time
  • 1pm GMT is still within sensible working hours for Brazil

Maybe send a doodle poll round

  • Alejandra is going to do this