Attendance

Attendees:

• Rolf Krahl
• Andy Gotz
• Louise Davies
• Marjolaine Bodin
• Kevin Phipps
• Malik Almohammad
• Alex de Maria
• Antony Wilson

Apologies:

• Allan Pinto

Agenda

Site Updates

SESAME

Doing good. Some updates, a publication using data from SESAME machines. Publication in data portal & public data. Need to discuss some things with Alex. ICAT+ - how to upload e.g. summary pdf via ICAT+?

RK: for ICAT, depends whether you want it in the same Dataset or a separate "documentation" Dataset.

MA: ICAT+ has some specific documentation about this I think.

AdM: after 90 days data is removed, e.g. summary, plots, to make them permanent we create a "gallery" folder/data, ingested & archived as usual. Also ingest into mongoDB so it's available always. Online folder, all data in here is moved to disk space so it's always available. Small files in "gallery", "online" folder for larger files. MongoDB vs disk space. Bypassing data removal from disk storage after 90 days.

HZB

ROCK-IT - Project inside Hermholtz institution. Coming to an end end of this year - relevant to ICAT is nexus creator software. Not finalised, but nexus creator called directly from blue sky instrument controls. Workflow from files from instrument into ICAT ingestion.

New projects on horizon. Battery data space. Help us to have better metadata & standardisation.

Sample database, SEPIA, still in development. Kind of in a good shape now.

Bad news: want to move ICAT to VMs. Facing difficulties with VMs. Ideally wanted migration this week but it's now delayed. Not ICAT related directly, more the VMs and podman.

ESRF

Big proposal to work on data in ESRF. If accepted work with metadata people in HZB, with SESAME and more. Won't know until the end of the year. Will encourage people to use open data, developments around this (metadata, ontologies, logbooks)

Tried to upgrade to ICAT v5, struggled so rolled back.

AG: maybe easier to upgrade to straight to v6?

KP & LD: yeah, mostly everyone else is on v6 so backports are tested less vigorously.

RK: I'm also on v5, upgrade after VM

New plotting widgets. On top of web UI adding visualisations. Phase out ispyb by 2026, finishing everything. Invite users onto ICAT from October. Stop ispyb by March 2026.

Happy with performance, difference between users and managers. Presumably it's rules, next few months need to look at these. Dockerise pointing to different DBs with different rules and compare & performance test.

RK: shut down ispyb - migrating old data?

AdM: no. stop using it for new experiments but old experiments archived. Could move old data, but would need extra resources. Unsure if there is motivation.

AG: better to leave it there. most users looking at recent data and keep old data just in case.

RK: downside need to keep ispyb running.

AG: die naturally at somepoint and export very old data to DB. won't keep it running forever

RK: that's the concern I have, where to store data for long time. people want data just in nomad, but I insist it must be in ICAT as nomad not necessarily running for 10+ years.

AdM: ispyb under data policy. No raw data in ispyb. Authentication done via proposal accounts and they'll eventually disappear as well. So will eventually be inaccessible.

ISIS

LD: Not much to report. ISIS in in cycle, there's a few pending jobs but are waiting until after cycle.

DLS

LD: continue to work on open data developments, it's picked back up again

KP: restore data to STFC data analysis platform. And improved Lucene indexing, access via API/CLI.

Component Updates

icat.server

6.2.0

6.2.0 snapshot? has it been tested? can it be released?

Alan Kyffin (via KP): hopefully soon

7.0.0

Schema updates, most likely pending on Rolf. Maybe out this year, but depends on workload.

KP: facilities need to evaluate breaking changes

RK: yes, that's why they were pushed back to v7

RK: sample <-> investigation, many to many means that sample needs a unique column, likely pid, but that means could find duplicates in existing facilities. will likely provide scripts to help this.

python-icat

New release corresponding to ICAT 6.2.0 release. Only minor changes though

datagateway-download-api (TopCAT java backend)

Some changes, but don't think anyone else is using this.

AOB

ALBA?

Haven't heard from them in a while - who do we know who works there? AM: Rodrigo still works there. Plus some other names

Antony Wilson

And old/new face, has been involved ICAT before but here as he's the STFC group leader. Antony is helping organise the F2F.

F2F

Everyone should have emails with proposed dates in January. Will be in Cosener's House in Abingdon - halfway between RAL and Oxford. Marjolaine & Rolf have said ok.

 

AdM: going to DLS in november, so might not go twice

MA: me and Salman interested in attending. Just logistics.

KP: you said you need a months notice so January should be fine

 

KP: Looks like it's smaller attendance, is it still going to be useful?

AW: free lunch or dinner. but no alcohol!

AdM: alternative to have a half day online?

KP: yes

AdM: F2F are tiring and more work but good on social side

RK: you can go a lot deeper in F2F rather than offline. In favour of a F2F. Need a question on what to discuss.

KP: roadmap etc.

 

RK: was concerned about room rates as Cosener's but seems reasonable

LD: there is a more budget hotel on other side of Abingdon if people's travel policies can't justify Cosener's

 

KP: so plan for January, will try and reach out to e.g. ALBA to try and get them to respond. Seems no preference for which days of the week. Plan is for a full day on the first day, and second day a half day.

KP: meeting not at RAL, but can organise trip to RAL for a tour.

RK: I've had a tour a while ago.

MB & MA: interested in a site visit.

 

KP: so we'll go ahead with planning. I'll chase people up, but do let people know if you talk to any ICAT people in the meantime.

Teams vs Zoom

Seems like teams is working fine so we'll delete the Zoom meetings for the future.

RK: can provide a Zoom link if something goes terribly wrong.