pandas version 3 release notes
Background
PANDAS version 3 is the latest version of the PANDORA Digital Archiving System developed by the National Library of Australia and released in June 2007. PANDAS is the web archiving workflow management system used by National Library staff and staff in PANDORA participating agencies to collect and manage the archiving Australian web resources.
PANDAS 3 represents a complete redevelopment of the system involving rewriting the code on the WebObjects development platform to improve system performance and to comply more closely with standard Java deployment. In addition to re-engineering the code a number of significant workflow and user interface improvements have been incorporated into this release. The following notes highlight new functionality and changes that moves the system beyond PANDAS 2.
Please note that this document does not replace the PANDAS 3 User Manual. It is intended to highlight important aspects of PANDAS 3 that you should know before commencing using the system. The PANDAS 3 User Manual is being drafted and, to help you, is being made available online while it is being put together. You can view it here.
It is important that you read these release notes before starting to use PANDAS 3. It is up to you whether you wish to get acquainted with the system by using it or if you wish to refer to the manual first. Please also note that this document and the PANDAS manuals will be subject to updating and revision. So, if you chose to print off any of these pages be aware that you should also refer back regularly to the online version to ensure you are using the most up-to-date version.
Logging On, Browsers and Navigating the User Interface
Logging On
When you log in to PANDAS 3 you will be logged on automatically at your highest access level. This will be noticeable and advantageous to Agency Administrators. You will no longer need to ‘temporarily change your access level’ as you had to do in PANDAS 2.
Navigating the User Interface
The first thing you will notice is the new interface which resembles the style of the PANDORA web pages utilizing the same colour scheme and background images. More importantly, the Menu bar at the left side of the screen is always accessible to allow you to navigate more readily around the application screens. The Home link (or clicking on the PANDAS banner) will take you to the main PANDAS page with your personal worktrays (see more about worktrays below).
Special note about using Back buttons and links : Unlike PANDAS 2, PANDAS version 3 does not open in a browser window that removes the browser navigation options. While you can use the browser's Back and Forward buttons in certain circumstances without any problem, please try to get used to using the navigation options within the PANDAS site as using the browser navigation may cause problems in certain instances. Navigation options include the always visible menu at the left of the screen and Back links, Close and Save buttons within the PANDAS screens.
Browsers
Unlike the earlier versions of PANDAS, PANDAS 3.0 can be used in either the Firefox or Internet Explorer browsers. It is a matter of personal choice which browser you use, but our recommendation is to use Firefox as the preferred browser. Firefox also allows you to added useful plugins to assist with the QA process (these are not related to PANDAS in anyway, but if you are using Firefox for PANDAS then you will more readily be able to make use of them for your QA work).
Clicking to Select and Set and Apply Selection
Be aware that you need to click buttons to 'add', ‘apply’ or 'set' selections from pick-boxes and to save settings. In some applications or websites you may be used to selecting an option and it automatically applies (this is the magic of JavaScript in action). Due to the different and inconsistent ways browsers apply JavaScript for PANDAS we have tried to avoid using JavaScript for this sort of functionality. So the system will generally require that you make a selection then click an 'add', 'apply' or 'update' or 'save' button to complete the process. As an example, when adding Subjects to a title, first you select the subject from the dropdown box (and you might think that is enough - but wait there is more you must do) - then you must click the Add button. When you do you will see the selected subject added to the screen with a Remove button next to it. You will find this sort of procedure is required throughout the system when editing records.
In some cases you can use the Enter button on your keyboard (for example when using the Advanced Search function. But this is best avoided when working with (i.e. editing) records and you should click on the appropriate buttons on the screen.
Worktrays
Worktrays to Manage Personal Workflows
Worktrays are a major innovation in PANDAS 3. The worktrays are designed to help you manage your workflows better by organising the titles you are responsible for into trays representing their status in the archiving process. So, a title will appear in a worktray according to it’s status (e.g. Nominated, Selected, Gathering, Processing). This will help prevent you losing titles in the system and will also help you organise your workflow. In addition to providing you with an overview or your work on titles, they also provide links directly to actions relevant to the status of the title (e.g. you can link directly to the Process screen from titles in the Instances in QA worktray or link directly to the gather schedule screen from titles in the Permission Granted worktray).
Worktrays are organised under the general workflow headings of Transferred, Selection, Permission, Gather, Preserve, Publish, Catalogue and (for Administrators) Reports. Within these broader headings the specific worktrays appear. For example under Preserve you will find Worktrays such as View Instances for Upload and View Instances in QA. Worktrays will only appear under the headings if there are titles to fill them. So, if you have no titles scheduled within the next month you will not have a Scheduled Titles worktray under the Gather group of trays.
Scheduled Titles Worktray
A useful innovation that the worktrays provide is the ability to see your titles in your Scheduled Titles worktray that are scheduled for gathering over the coming month (e.g. if today is 27 March you will see titles scheduled from today until 26 April).
Worktrays by default will show the top 6 titles but you can click on the view All link to see all the titles listing in a worktray. There are also options to Expand All and Collapse All worktrays. The worktrays also have a Refresh option which you can use to make sure your view of any given worktray is updated. Titles in the worktrays are sorted top to bottom by the date relevant to the worktray, or example, by permission granted date in the Permissions Granted worktray and by gathered date in the Instances in QA worktray. If there is no obviously more relevant date then they will sort by registered date.
Administrators' Worktray Options
Agency Administrators since they have higher access levels than Standard Users are able to select either their Personal Worktray view (i.e. a view of just the titles they own) or an Agency worktray view. The latter allows Agency Administrators to view all the titles for their Agency irrespective of the owner.
The Nominated Titles Worktray
Titles in the Nominated Worktray are viewable by all users in the Agency. The Nominated Worktray indicates the user who nominated the title and the current owner. For agencies with a number of staff working on selecting titles for archiving, this allows staff to register and nominate titles for selection into a common pool of titles and then for any other staff to 'pick up' a title to Select it and commence work on it.
Title Records and Title 'Ownership'
View Title Screen
The View Title screen for any record gives you a good overview summary of the information recorded about a title without having to click through a number of screens. This is organised into boxes for Title Details (URLs, current owner, current status, date of registration, notes etc), Permission and Publisher Details (publisher type, URL, permission status and date), and Gather Details (method, last and next gathered dates, schedule type). If there are access restrictions associated with the title this will also be visible in a Restrictions box. There are links to view more detail, such as gather options and also links to edit the various parts of the title record.
All Titles are 'Owned' (by Default or by Choice)
A new concept introduced in PANDAS 3.0 is that titles will always have an owner associated with it. This is a necessary aspect of the worktrays functionality. Ownership is assigned to titles initially automatically and then through the transfer process. When a title is registered on PANDAS the user name of the user registering the title is recorded as the ‘owner’ (and also as the ‘nominator’ if the status is left as Nominated). If a user Accepts a title (i.e. Selects a title) from the Nominated Worktray they are automatically assigned ownership. In other words, the user who first Selects a title will automatically become the owner of the title. After that, transfer of ownership is managed through the Transfer functionality (see below). This means that other authorised users can still amend details to title (e.g. add notes) but this does not mean you automatically take over the ownership!
By clicking on the View Owner History link on the View Title screen you can see the ownership history associated with title.
Publisher and Permissions Details
Recording Communication with Publishers
You will be able to record, in a standardised way, communication contact events in PANDAS 3.0. You can add a new contact event and choose the type of event from a drop down list (e.g. enquiry, initial request), record the date and add notes.
Clear Step-by-Step Workflow for Recording Permission
The Title Edit - Permissions screen for the Title record has been designed to facilitate clearer step by step workflow for recording permissions with Step 1 'selecting the the title publisher', Step 2 'recording contact people', Step 3 'record communication' and Step 4 'update permission status' clearly set out.
Click to Create Emails
You can also open a new email message directly from the View Title or Title Edit - Permission screen. In addition, if your agency's form letters are online you set up a link to these which will appear in the left hand menu so you can copy and paste the text into the new email message. (Setting up a link to your form letters is something Agency Administrators can do through the Edit Partner Agency management screen.)
URLs, URLs, URLs
Seed URL and Title URL
Two URLs with different purposes can be recorded in the Title record. When registering a title the first URL box to fill in will be the Seed URL, which is the starting URL for the harvest. There is a second URL box for Title URL. This is the URL for the title which may or may not be the same as the URL you want to use to start the gather with. In many, perhaps most cases, it will be the same as the Seed URL. So, if the Seed URL and Title URL are the same, when filling in the Title details you only need to put the URL in the Seed URL box and when you click on (or tab to) another option or box to complete on the Title Edit screen the Seed URL will automatically default into the Title URL box. If the Title URL is in fact different then you can change it. The Seed URL also defaults to the Gather Details screen, so you don't have to re-enter it there.
Publisher URL
It is also possible to record another URL as the Publisher URL. This is associated specifically with the publisher and might be used to link to the publisher's main page or contact details page etc. As with PANDAS 2 you can create new publisher records when you create new title records, so you can add a Publisher URL at the time you create the Publisher record. You can also Edit publisher record from the Title Edit - Permissions screen, so you can also add a Publisher URL at any time if you are selecting an existing Publisher to add to a new title.
Searching
There is now a “quick search” option as well as an advanced search options. The quick search is a single search box that appears at the top right side on all PANDAS screens. You can use this option to quickly search for titles and PI’s. This quick search option uses a character string default for searching (not Boolean searching). This is the same sort of default that was used in PANDAS 2. So, for example, you could search for the title 'journal of engineering' with the character string: al of eng
The advanced search option uses as the default the Boolean AND logic based on whole words. So to search on the title 'journal of engineering' you could use the terms: journal engineering or engineering journal, but not al or eng.
The advanced search also allows you to combine search terms across different fields such as title name and publisher; and also allows many ‘limiting’ options such as format, owner, subject, registration date. This allows for combination of search parametres that can produce report-like search results.
Transferring Titles
The PANDAS 2 title transfer and assign ownership functions have been combined into the Transfer functionality in PANDAS 3. This means titles can be easily transferred to between people either within the same agency or to another agency. Transferred titles appear in your Transferred worktray. The person transferring the title can stipulate that the transfer is “acknowledged” by the recipient. This means that an Accept link will appear next to the title in the Transferred worktray. The recipient must click on this to acknowledge receipt. But note that this does not (as you may suppose) send a receipt back to the sender; the purpose of this acknowledge/accept functionality is so that the sender can be sure that the title appears in the Transferred worktray of the recipient and that the recipient can see a transferred title in the Transferred worktray before it moves to the worktray associated with the title’s status (e.g. if it is a nominated title it will go to the recipient agency’s Nominated Worktray). The acknowledge/accept functionality also ensures that the recipient sees any attached transfer note.
How Many Instances in One Day?
PANDAS 3 introduces time stamping of instances not just date stamping. In PANDAS 2 you could only harvest one instance for a title in a day as the instances were recorded with an eight digit date stamp, i.e. ddmmyyyy. In PANDAS 3 the time stamp is being added to the date stamp so that instances can now be recorded with 12 digits including 4 digits for the hour and minute, e.g. ddmmyyyy-hhmm
This means you can harvest and archive the same title a number of times within a day (logically that is up to times a day though I don't suggest anyone to to that extreme!).
Gather Details and Gather Settings
In PANDAS 3 the commonly used gather settings such as the gather filters, extra URLs and username/passwords (together with the gather method and scheduling of course) are distinguished from other settings by being presented on the Gather Details screen. The remaining gather settings options are found on the Gather Options screen along with an HTTrack command line summary of all gather settings. The HTTrack command line summary tells you, in coded form, what is actually being sent to the gatherer (you can refer to the HTTrack User Guide online if you want to interpret this). The Gather Options page also allows you chose a predetermined gather Profile. Currently there are only two profiles, the PANDAS default profile and the HTTrack default profile. However, if in time we establish other profiles these could be used to change gather settings in a simple way without the need to know the HTTrack codes. For now you will not need to change the profile. For most of the titles you work with you should only need to use the options on the Gather Details screen.
One new functionality that is introduced to the Gather Details screen is the option to add some Preset filters. This allows you to add a standard set of filters to, for example, exclude everything except image and style related files. These preset options can be used as a basis to build up the specific set of filters you need for the title you are working on.
Uploading Title Instances
Uploaded title are what were referred to in PANDAS 2 as 'publisher supplied' gathers. There is a clearer distinction in regard to upload titles with titles set to Upload having their own worktray. Upload titles will now also be put through the post-processing. This means that you upload titles in much the same way as you did in PANDAS 2, but when you have uploaded the files you then click the Post Processing button to run the upload through the same scripts as are normally applied to harvested titles. This allows, for example, uploaded titles to have preservation metadata gathered and for external links to be re-written to point to the PANDORA message alerting users that the link they have clicked on has not been archived.
Harvesting Problems
QA system uses the JIRA issue tracking software. Problems are logged through the PANDAS user interface and are sent by the system to JIRA where NLA staff can manage the problem response. The problem description and response are still viewable through the PANDAS UI.
Subject Listings
Subject listings can now be added/deleted/edited through the PANDAS 3 user interface. This can only be done by users with PANDAS Administrator access – so it is not a free-for-all! However it will enable us to work towards the long overdue task of adding more subjects to the PANDORA browse options.
Issues Relating to Migration from PANDAS 2 to PANDAS 3
‘Old’ titles may (will!) appear in your worktray – this will be because of an un-resolved status. For example no permission recorded. Agency Administrators can see all such titles for their agency. These should be resolved. Remember this is part of the function of the worktrays, to alert you to outstanding tasks.
Titles Requiring Cataloguing
You are also likely to see quite a lot of titles in your Titles Requiring Cataloguing worktray! This will be because you have not added a Libraries Australia (ANBD) catalogue record number to the PANDAS record and this may be because you have not in fact catalogued the record or because you have forgotten to update PANDAS with the ANBD catalogue record number. The action you need to take is to update the ANBD number field in the PANDAS record with Libraries Australia number (so you may have to catalogue the record first!). If there is a valid reason for the title not to be catalogued (e.g. it might be part of a collection that is only catalogued at the the collection level) you can tick the Cataloguing Not Required checkbox in the PANDAS record and this will remove it from the Titles Requiring Cataloguing worktray.
Migration of Gather Settings
The migration of data from the PANDAS 2 database to the PANDAS 3 database is a complex and difficult matter and the migration of gather settings is particularly complicated given a number of changes to the settings presented between PANDS 2 and PANDAS 3.
All the gather settings as found on the PANDAS 3 Gather Details screen (that is, gather filters, extra URLs and username and passwords) will be migrated from PANDAS 2 to PANDAS 3. In respect to the rest of the settings (those found on the Gather Options screen) it is our intention to apply the standard PANDAS 3 default gather settings profile as the basis for all migrated titles. However one or two of the fields (those that have been more commonly changed and which have major implications for gathers) will be migrated from the PANDAS 2 data, most certainly, for example, the maximum mirror depth setting.
