A few days ago, I asked the following question in the Mashcat Slack: “if you’re a library data person, what questions do you have to ask of library systems people and library programmers?”<\/p>\n
Here is a question that\u00a0Alison Hitchens asked based on that prompt:<\/p>\n
I’m not sure it is a question, but a need for understanding what types of data manipulations etc. are easy peasy and would take under hour of developer time and what types of things are tricky — I guess an understanding of the resourcing scope of the things we are asking for, if that makes sense<\/p><\/blockquote>\n
That’s an excellent question \u2013 and one whose answer\u00a0heavily depends on the particulars of the data change needed, the people requesting it, the people who are to implement it, and tools that are available. \u00a0I cannot offer a magic box that, when fed specifics and given a few turns of its crank, spits out a reliable time estimate.<\/p>\n
However, I can offer up a point of view: asking somebody how long it takes to change some data is asking them to take the measure of\u00a0their confidence and of their constraints.<\/p>\n
In this post I’ll focus on the matter of confidence. \u00a0If you, a library data person, are asking me, a library systems person (or team, or department, or service provider), to change a pile of data, I may be perfectly confident in my ability to so. \u00a0Perhaps it’s a routine\u00a0record load that for whatever reason cannot be run directly by the catalogers but for which tools and procedures already exist. \u00a0In that case, answering the question of how long it would take to do it might be easy (ignoring, for the moment, the\u00a0matter of fitting the work onto the calendar).<\/p>\n
But when asked to do something new, my confidence could start out being quite low. \u00a0Here are some of the questions I might be asking myself:<\/p>\n
Am I confident that I’m getting the request from the right person? \u00a0Am I confident that the requester has done their homework?<\/strong><\/p>\n
Ideally, the requester\u00a0has the authority to\u00a0ask for the change, knows why the change is wanted, has consulted with the right data experts within the organization to verify that the request makes sense, and has ensured that all of the relevant stakeholders have signed off on the request.<\/p>\n
If not, then it will take me time to either get\u00a0the requester to line up the political ducks or to do so myself.<\/p>\n
Am I confident that I understand the reason for the change?<\/strong><\/p>\n
If I know the reason for the change \u2013 which presumably is rooted in some expected benefit to the library’s users or staff \u2013 I may be able to suggest better approaches. \u00a0After all, sometimes the best way to do a data change is to change no data at all, and instead change displays or software configuration options. \u00a0If data does need to be changed, knowing why can make it easier for me to suss out some of the details or ask smarter questions.<\/p>\n
If the reason for the change isn’t apparent, it will take me time to work with the requester and other experts and stakeholders until I have enough understanding of the big picture to proceed (or to be told to do it because the requester said so \u2013 but that has its own problems).<\/p>\n
Am I confident that I understand the details of the requested change?<\/strong><\/p>\n
Computers are stupid and precise, so ultimately any process and program I write or use to effect the change has to be stupid and precise.<\/p>\n
Humans are smart and fuzzy, so to bring a request down to the level of the computer, I have to analyze\u00a0the problem until I’m confident that I’ve broken it down enough. Whatever design and development process I follow to do the analysis \u2013 waterfall, agile, or otherwise \u2013\u00a0it will take time.<\/p>\n
Am I confident in the data that I am to change?<\/strong><\/p>\n
Is the data to be changed nice, clean and consistent? \u00a0Great! It’s easier to move a clean data set from one consistent state to another consistent state than it is to clean up a messy batch of data.<\/p>\n
The messier the data, the more edge cases there are to consider, the more possible exceptions to worry about \u2013 the longer the data change will take.<\/p>\n
Am I confident that I have the technical knowledge to implement the change?<\/strong><\/p>\n
Relevant technical knowledge can include knowledge of any update tools provided by the software, knowledge of programming languages that can use system APIs, knowledge of data manipulation and access languages such as SQL and XSLT, knowledge of the underlying DBMS, and so forth.<\/p>\n
If I’m confident in my knowledge of the tools, I’ll need less time to figure out how to put them together to deal with the data change. \u00a0If not, I’ll need time to teach myself, enlist the aid of colleagues who do have the relevant knowledge, or find contractors to do the work.<\/p>\n
Am I confident in my ability to predict any side-effects of the change?<\/strong><\/p>\n
Library data lives in complicated silos. Sometimes, a seemingly small change can have unexpected consequences. \u00a0As a very small\u00a0example, Evergreen actually cares<\/i> about the values of indicators in the MARC21 856 field; get them wrong, and your electronic resource URLs disappear from public catalog display.<\/p>\n
If I’m familiar with the systems that store and use the data to be changed and am confident that side-effects of the change will be minimal, great! If not, it may take me some time to investigate the possible consequences of the change.<\/p>\n
Am I confident in my ability to back out of the change if something goes wrong?<\/strong><\/p>\n
Is the data change difficult or awkward to undo if something is amiss? \u00a0If so, it presents an operational risk, one whose mitigation is taking more time for planning and test runs.<\/p>\n
Am I confident that I know how often requests for similar data changes will be made in the future?<\/strong><\/p>\n
If the request is a one-off, great! If the request is the harbinger of many more like it \u2013 or looks that way \u2013 I may be better off writing a tool that I can use to make the data change repeatedly. \u00a0I may be even better off writing a tool that the\u00a0requester<\/em> can use.<\/p>\n
It may take more\u00a0time to write such a tool than it would to just handle the request as a one-off, in which case it will\u00a0take time to decide which direction to take.<\/p>\n
Am I confident in the organization?<\/strong><\/p>\n
Do I work for a library that can handle mistakes\u00a0well? \u00a0Where if the data change turns out to be misguided, is able to roll with the punches? \u00a0Or do I work for an unhealthy organization where a mistake means months of recriminations? Or where the catalog is just one of the fronts in a war between the public and technical services departments?<\/p>\n
Can I expect to get compensated for performing the data change successfully? Or am I effectively being treated as if\u00a0I\u00a0<\/em>were the stupid, over-precise computer?<\/p>\n
If the organization is unhealthy, I may need to spend more time than ought to be necessary to protect my back \u2013 or I may end up spending a lot of time not just implementing data changes, but data oscillations.<\/p>\n
…<\/p>\n
The pattern should be clear: part of the process of estimating how long it might take to effect a data change is estimating how much confidence I have about the change. \u00a0Generally speaking, higher confidence means less time would be needed to make the change \u2013 but of course, confidence is a quality that cannot be separated from the people and organizations who might work on the change.<\/p>\n
In the extreme \u2013 but common \u2013 case, if I start from a state of very low confidence, it will take me time to reach a sufficient degree of confidence to make any time estimate at all. \u00a0This is why I like a comment that Owen Stephens made in the Slack:<\/p>\n
Perhaps this is part of the answer to [Alison]: Q: Always ask how long it will take to investigate and get an idea of how difficult it is.<\/p><\/blockquote>\n
In the next post, I discuss how various constraints can affect time estimates.<\/p>\n
Share this:<\/h3>