Crossref updates its POSI self-assessment after 1,005 days.

Disclosure

I was a co-author of the original Principles for Open Scholarly Infrastructures. I also worked for Crossref and wrote its first and second POSI self-assessments. However, I’ve based this analysis on public information.

TL;DR

After 1,005 days, Crossref updated its POSI self-audit. Crossref has made some important progress in increasing board diversity, establishing financial reserves, removing a closed source technical dependency from its system, and ensuring that IP is not a barrier to forking Crossref services.

This is good news. But I remain concerned.

The report has some important omissions and transparency issues. It fails to explain why Crossref took nearly three years to update its POSI assessment and does not mention a significant regression in its POSI commitments during that time.

The report also does not address some important, unresolved questions like “Is it possible for a membership trade association like Crossref to be ‘stakeholder governed’ if its stakeholders are not all members of ‘the trade’ in question?”

I’ll provide a more detailed list of my concerns below.

But, before I do that…

On stakeholder critiques

In a previous post I made reference to the point made by IOIs Katherine Skinner about the research community’s reluctance to publicly discuss the challenges of open scholarly infrastructures.

Often we’re reluctant because we’re afraid that we might undermine confidence in already weakened services and that our criticisms might make matters worse.

This isn’t something that we have to worry about with Crossref.

Sometimes we worry about creating headaches for past colleagues, or about alienating potential future colleagues.

I do worry about that.

And, in this case, I also worry that, by critiquing Crossref’s self-audit update, I’ll be discouraging other POSI adopters from revising theirs for fear of similar treatment.

Or worse – that this critique will discourage other organizations from adopting POSI in the first place.

One of my proudest moments was when the Crossref board voted to adopt POSI in 2020.

As I write this, 22 organizations have adopted the principles.

I want to see POSI thrive. I want to see the POSI Posse thrive.

But it would be a shame if these self-audits become another phatic managerial communication ritual – like issuing vague, unactionable, and unmeasurable mission statements.

Or boilerplate bromides designed to lull people into complacency.

Because that will alienate stakeholders. And undermine the value of POSI endorsements.

If other POSI self-audits follow the pattern set by this one, I’m afraid that is exactly what will happen.

Omissions, transparency issues, and open questions

The most obvious omission from Crossref’s 2024 self-audit is that it doesn’t answer these questions:

Maybe Crossref was unaware of the blog post I wrote a week before, marking 1,000 days since it had last updated its POSI self-audit. Or of the subsequent reaction to the post on social media. Or maybe Crossref was aware, but did not want to draw more attention to it. Or maybe Crossref was aware, and just didn’t care.

Regardless, this suggests Crossref hasn’t fully internalized what it means to be “open.”

“Openness" implies a willingness to share information transparently. Even when that information is inconvenient or embarrassing. Doing this shows integrity– by demonstrating readiness to accept responsibility and address concerns without taking offense or being defensive.

And, even without explicitly drawing unwelcome attention to my post, Crossref could have at least addressed the broader concerns about the lengthy delay and what it says about Crossref’s regard for stakeholders.

Given the attention the post gained on platforms like Mastodon and Bluesky, it’s clear these issues resonate with Crossref’s community.

Had Crossref said something as simple and open as:

“We’re sorry it took so long; we should have updated our POSI self-audit sooner. Our criteria for updates are X and we’ll update you every Y days from now on”

It would have been enough. It would have shown integrity, and Crossref stakeholders would likely be happy to move on to other issues.

But it didn’t.

And Crossref can’t use the excuse that nothing of significance happened during those 1,005 days.

Stakeholder governance

In 2021 the Crossref membership elected a funder to the Crossref board of directors, but the funder stepped down mid-term and the nominating committee chose to fill its seat with another publisher. This means that, in 2023, Crossref regressed into non-compliance with stakeholder governance, a critical POSI commitment. Crossref did not report this at the time, and it fails to mention it in the 2024 update, instead stating that:

“We’ve been yellow and we’re still yellow…”

…implying continuity between the 2021 and 2024 self-audits when the reality was there was a regression.

That is not how self-audits are supposed to work and presenting the sequence of events this way shows a worrying lack of transparency and accountability.

Crossref explains that it still has not met the commitment to “stakeholder governance” because:

…being fully stakeholder-governed is challenging

This is true, but the 2024 self-audit doesn’t really explain why.

What everybody knows as “Crossref” is the trading name of the Publishers International Linking Association (PILA). PILA is a non-profit, but PILA is not a “charitable organization” (a “501(c)(3)” under the US tax code), instead it is a “business league” (a 501(c)(6) under the US tax code). This has important implications for governance because only “members” can govern Crossref and Crossref membership is restricted to those “of the same trade, business, occupation, or profession.”

While Crossref has always been admirably catholic in its definition of what constitutes “a publisher” (for example, funders can be Crossref members because they “publish” grants), there will always be many Crossref stakeholders who are not in “the trade,” no matter how much Crossref stretches the meaning of “publisher.”

Crossref metadata consumers are notably excluded from this arrangement. Initiatives like the Barcelona Declaration highlight infrastructures such as Crossref as essential to open research evaluation, underscoring the need for metadata consumers to be represented in Crossref’s governance.

There are also certain kinds of stakeholders who cannot join membership organizations of any type. Some government and intergovernmental agencies, for example.

Which raises some questions:

Create financial reserves

To be fair, Crossref didn’t just bury bad news. It also buried good news.

In the 2024 update, Crossref mentions that in 2023, it completed the creation of its contingency fund.

In 2023.

And apparently Crossref thought this also didn’t warrant updating stakeholders.

Ask any POSI adopter which commitment is going to be the hardest for them to meet, and they will likely flag this one.

It commits POSI adopters to create a contingency fund that covers operating expenses for a long enough period to allow for an orderly wind-down of the organization and the infrastructure that it runs.

Crossref, by almost any measure, is one of the largest and most financially stable of the organizations that have adopted POSI, and meeting this commitment was a stretch even for them.

So you would think meeting the commitment to create a financial reserve would be something to tell its stakeholders and to celebrate.

Over a year ago. When it happened.

When we drafted POSI, we knew meeting this commitment would be a difficult hurdle but hoped it would spark some discussions about how meeting the commitment could be made less onerous, short of watering it down.

There is nothing in POSI that says that each organization has to raise these funds on its own.

This is a good example of where collective action could help instead.

Open Source

In 2024, Crossref made an important move from the proprietary Oracle database to the open-source PostgreSQL. This is, as Crossref notes, an important step in its quest to be able to release all of its code as open source – a key insurance provision in POSI.

In its 2022 self-audit, Crossref highlighted that the move to open source was likely to be a multi-year effort.

But the 2024 self-audit does not provide an estimated timeline.

Maybe it hasn’t changed.

But without any sort of rough timeline, it is hard for stakeholders to understand whether or not Crossref is making reasonable progress on this commitment.

The Crossref self-audit update describes the move off Oracle and onto the open-source database PostgreSQL as:

…an important step towards paying down technical debt and moving the system fully into the cloud.

Which is reasonable. But it does raise an obvious question. Cloud services offer proprietary functionality that can lock in users in almost exactly the same way that proprietary databases like Oracle can.

And since we are discussing dependencies, it is probably also worth asking about a few other not-so-obvious dependencies that affect Crossref.

The first is an organizational dependency.

Crossref is a DOI registration agency. It is dependent on three other organizations that have not endorsed POSI. Namely:

• The DOI Foundation
• The DONA Foundation
• Corporation for National Research Initiatives (CNRI)

The second is a technical dependency.

The DOI system is based on the Handle technology stack developed by CNRI. The Handle system license is not OSI-approved.

Transparent governance

Crossref doesn’t talk about transparent governance in the 2024 self-audit, but it should have.

After adopting POSI, Crossref added new sections to its website to provide stakeholders with transparent information about its governance, operations, and finances. These sections are an excellent resource, and everything I’ve highlighted here can be gleaned from these pages.

The board and governance page, for example, includes links to the latest “board motions passed.” Even before adopting POSI, Crossref made a habit of updating this page with motions from recent board meetings – typically within a few days of the meeting.

This is also where you can find motions related to POSI, such as this one passed as part of the nominating committee’s instructions during the November 2023 meeting:

Encourage and recruit Crossref members who are research funders to apply to the Board.

Crossref has three board meetings a year- in March, July, and November.

The last board motions listed are from the March 2024 meeting. This means that board motions from the two board meetings since March 2024 are missing.

What next?

First, I want to encourage those who have adopted or are about to adopt POSI to make their self-audits as open, honest, and self-critical as they can.

Stakeholders will be forgiving if you are having a hard time meeting the POSI commitments and transparently name those challenges. But they won’t be if it looks like you are trying to hide something.

The only way we are going to educate stakeholders about the challenges involved in building and running open scholarly infrastructures is if we are candid. And we need educated stakeholders to support the collective action required to support open scholarly infrastructure.

And, while I think self-audits are important for educating and building trust with stakeholders, I don’t think they are enough on their own.

Even professional external audits can fail spectacularly.

Self-audits are, well… self-audits.

It’s not that we should suspect that self-audits are inherently dishonest.

However, self-audits rely on organizational self-awareness and a culture that supports self-criticism.

All organizations have ingrained internal ideologies, so all organizations have blind spots.

And many organizations have “third rail” topics that everybody in the organization is afraid to touch.

One problem with most audits is that very few people outside the audited organization have the information required to scrutinize the audit effectively.

But this isn’t true of POSI.

The themes and individual objectives outlined in POSI are focused on openness. These are things that stakeholders can verify. Here is a small sample:

• Do they have a governance structure that reflects all the infrastructure stakeholders?
• Can you examine their board and committee documents?
• Can you look at their publicly available financial statements? Do they make sense?
• Is their source code open? Can you actually do anything with it, or is it a tangled, undocumented mess?
• Do their operations depend on source code or other organizations that are not open?

So I want again to encourage stakeholders to scrutinize the POSI self-updates of organizations they rely on. We increasingly depend on these organizations; if they fail or become enclosed, it will affect virtually all research stakeholders.

I particularly want to call on funders and policy makers to take an active role in scrutinizing these self-audits.

The open research initiatives they are funding are dependent on there being an open scholarly infrastructure to build on.

I look forward to seeing the other POSI Posse self-updates.

I’ll leave it to other stakeholders to step in and scrutinize them.

I’ll focus instead on those who haven’t yet endorsed POSI.