Attendance

Attendees:

• Rolf Krahl
• Andy Gotz
• Louise Davies
• Marjolaine Bodin
• Kevin Phipps
• Malik Almohammad
• Santhosh Anandarama
• Alan Kyffin

Agenda

Site Updates

HZB

Nothing to report

ISIS

Production release - updating DOI metadata with instrument PIDs. Plus fixing other minor issues, patching machines etc.

Currently working on new backup service

DLS

Working towards September release of DOI functionality. DLS ingest - working on tagging raw data.

AG: won't published processed data?

KP: current plan. DLS archive lots of mixed stuff, user processed data all in there, so taking cautious approach. View to when it's up and running to tag other types of data.

LD: idea to get something working first, then when it's used if comes critical to users, releasing other data priority.

KP: pressure on DLS to get data open and available.

AG: DLS is working with PDB on automatic uploading of structures. Open the data, link to raw data?

KP: hopefully people see benefits of opening data

SESAME

Maintenance - upgrade frontend. 2FA. working with data acquisition team on DOI minting.

Ask users to create a report describing the data, report is made into DOI landing page. Different from ESRF - ESRF mint directly after proposal approval.

MB: end of experiment. mint automatically. users have to submit report, but users have few months to do this.

MA: where get data description?

MB: from user portal, abstract etc.

MA: tried asking for that, but got told better to have users generate reports after experiment

AG: automatically minted DOI - users got told this is what they should cite. However whole session sometimes not useful. Has caused issues - people cite session DOIs rather than specific DOI. Trouble with replication, which data is it. At user meeting earlier, told users they must publish DOIs

MA: user go to data portal and do it themself?

AG: yes, we allow anyone in experimental team to mint

MA: DB complicated, try to make it easier for users. Users asked to mint ICAT DOIs, but users want to add users not in DB. We have to add. Best practice need to decide.

AG: you can add anyone you want to DOI. even if not in our DB. In future, we should allow any user to create DOI.

MA: another layer of checking DOIs to ensure users mint in the right way.

AG: we don't do this - we currently get few mistakes. Suggest minimum number of characters for abstract. And trust users - sometimes users demanding the DOI very short notice.

MB: a couple of tickets to fix typos, but otherwise works well

RK: journal article should be based on processed data not raw data. not necessarily in our ICATs

AG: we do now have processed data

RK: yes, but not necessarily linked to raw data. ticking raw data for a DOI is maybe not useful for users, data journal is based on is data not in ICAT.

AG: some journals/users prefer raw data, readers can go back to original data.

MA: we ingest processed data - trying to push it to be used by users for all data access.

RK: often journal data is bespoke for that journal. ideal case, have full provenance chain.

AG: synchrotron part may only be small part of source of data for journal. maybe they don't want to use synchrotron data portal for other data sources. we want only to store data derived from our data.

RK: maybe different use case, we have data from other sources, e.g. computer simulations.

LD: on adding external users, for DLS we allow it via adding ORCiDs

ESRF

New tomography domain portal, link: https://cultural-heritage.esrf.fr/tomo/

Working on another portal for diffraction

Working with external company on improving UI for domain portals

Instrument PID landing pages: e.g. https://doi.esrf.fr/10.15151/ESRF-INSTR-279G

Still fighting with performance issue with ICAT. Increased memory and changes some Oracle DB parameters, also refactored/simplified some queries.

AG: regular crashes, 1 or 2 times a week. not identified source. maybe we're hitting a bottleneck as we have many users during experiments. different roles have different permissions. e.g. some users have 2 seconds per request, admin is 0.5 secs per requests.

KP: different performance for different users is based on rules. can enable settings to see queries going to DB, and see some requests are way more complex than others. Patrick has had thoughts on improving rules/performance, but nothing easy/obvious.

AG: dropping rules.

LD: rule order, rules are tried in ascending length of what field.

 

AG: continue to improve technique specific displays

 

AG: elettra is looking as same software stack to replace ispyb. no other backup plan!

KP: there used to be someone from elettra in these meetings? they wanted a data archive but no money

AG: apparently they're using ICAT: https://zenodo.org/records/22122473

 

RK: instrument PIDs, project within DESY within OSCARS.

AG: yes, aware of them, met with them. Also met with ISIS. current landing pages are WIP, defining what a beamline is etc.

RK: PIDFest in October, some people making white paper on Instrument PIDs

Component Updates

icat.server

ICAT 7

Investigation sample work - is that all that's pending for ICAT 7? No, other schema changes, Sample PIDs. Not yet done

Patrick doing some indexing work on the Investigation <-> Sample changes.

Containerisation

Difficulty with config files, open war, edit run.properties, repack...

Option to use microprofile config. Standard, in all runtime environments. Would be able to then use environment variables, but still backwards compatible. Do that on all components.

RK: big change for some components, lots of config. e.g. oaipmh

AK: yes, but microprofile config is supported by everything. multiple ways of supplying config - should be easy to add support for existing run.properties files.

AK: I'll write it up and put it on the mailing list to give more details.

icat.lucene & datagateway-download-api

Minor release to fix issue on ISIS - Patrick's new path parsing code only supported UNIX path separators, ISIS uses windows.

Also fix issues in datagateway-download-api with not specifying a mail server crashing the service & ISIS downloads (aka single level IDS) being reported as 0 B.

AOB

Postgres?

Investigating it at ESRF.

RK: interested

AK: going to use it for new facility EPAC. setting it up, but no issues so far.

AG: if have questions can post in mailing list.

RK: SEPIA based on postgres. Not in prod, but prototype runs fine.

KP: postgres is in prod, but postgres DBs run on VMs, Oracles bare metal. Our DBs too big for VMs, but future will likely resolve this.

AG: Oracle VMs for our DBs - maybe source of performance issues!

RDA Plenary

RK: RDA Plenary in London in October.

AG: interested but not sure if have the bandwidth

RK: early bird prices end today

KP: normally no one in our group goes, but others from our department will likely go.