Operational Plan 2019-20
TIES Sub project: Reading System Evaluation
Submitted by: George Kerscher & Avneesh Singh
Last updated on: September 26, 2019
Table of contents
Reading Systems Accessibility Screening Methodology Tool
Nature of activity: part of the TIES Chartered project
The goal of the reading systems accessibility Screening Methodology project is to encourage the development of accessible reading systems by providing the accessibility evaluation methodology which is accepted as the de facto benchmark for evaluating accessibility of EPUB reading systems in the industry by DAISY members, disability groups, education system & governments.
The activity is a sub project under the TIES project (Transition to Inclusive EPUB 3 eco System). The over arching objective of the sub project is to increase the worldwide availability of the accessible EPUB 3 reading systems.
The Reading System Accessibility Screening Methodology was originally a white paper developed by the DAISY Consortium in conjunction with Technology For All (TFA) and distributed to the DAISY Community in June 2013 in Copenhagen. The question was how to best take this methodology and implement it in a way that will have the most worldwide impact.
DAISY Consortium collaborated with BISG and IDPF for creating an area for Reading System Accessibility Screening on http://www.epubtest.org/ where the main stream testing for features of EPUB 3 reading system was hosted.
In early 2017 IDPF combined with W3C, therefore the ownership of the domain epubtest.org has shifted to W3C. So, it is now a collaboration between the DAISY Consortium and the W3C, where the domain is owned by W3C and the website is hosted on a DAISY server to have a better control of the testing methodology.
The project will have the following overarching objectives:
· Further development & maintenance of the methodology, accessibility test suite & web interface. The 2020 work items will include continued work on internationalization of website and optimization of the accessibility testing user interface. To ensure that the community has the up-to-date evaluations of reading systems, we will also maintain a “Roundup of the Reading Systems” for non-technical users. Feedback is provided to respective developers like the Readium foundation & EDRLab, and other open source & commercial reading systems developers.
· Acceptance of the methodology. The methodology should be accepted as de facto benchmark for evaluating accessibility of EPUB reading systems. The methodology and the test results should earn credibility from reputed universities, major players in Mainstream ICT Providers, DAISY members, and related government agencies.
Milestones are the concrete deliverables set forth on the basis of confirmed resources (either we have staff or we have confirmed funds and/or resource commitments).
Priorities are the deliverables based on external contributions, our influence on others, available opportunities etc.
· Keep track of the feedback for the testing methodology & web interface.
· Develop methodology & accessibility test books for advance accessibility features.
· Update the methodology, web interface of testing website & the test books based on the feedback of evaluators, the suggestions from the experts in the community, and new versions of digital publishing specifications (average interval of the updates should be 6 months).
· Maintain French localization of the website and accessibility test suite.
· Establish a stable group of volunteer testers.
· Communicate the short comings of reading systems to respective technology developers.
· Update the narrative “Roundup” document for accessible reading apps at interval of three months, & promote it in DAISY community, educational institutions (universities & schools), W3C publishing groups, BISG etc.
· Keep track of the feedback, evaluate the effectiveness of the system once a year and refine the further plans.
· Develop methodology & accessibility test books for advance accessibility features
· Further expand the focus on disabilities other than blindness, low vision & dyslexic, and develop methodology & accessibility test books further to incorporate their essential needs.
· Update the methodology, web interface of testing website & the test books based on the feedback of evaluators, the suggestions from the experts in the community, and new versions of digital publishing specifications (average interval of the updates should be 6 months).
· Expand the established group of volunteer testers
· Communicate the short comings of reading systems to respective technology developers.
· Update the “Roundup” narrative document for accessible reading apps at an interval of 3 months & promote it in DAISY community, educational institutions (universities & schools), W3C publishing groups, BISG etc.
· Consider expanding accessibility testing beyond EPUB 3 reading systems to reading inside learning management systems (LMS).
· Evaluate at least 40 reading systems via contributions from organizations & individuals (by also using in-house training & support team when required). These should also include the re-evaluations performed on the new versions/models of reading systems which were already evaluated.
· The methodology is recognized as suitable approach for EPUB 3 reading systems accessibility evaluation by DAISY members, industry, disability groups, education system & related departments in governments. It includes getting endorsements from influential personalities & entities in education, acknowledgment from publishing & technology industry and recognition from community leaders.
· Identify the reading systems that provide best accessibility and usability and support them in marketing their accessible product along with DAISY Consortium’s testing methodology.
· Explore how epubtest.org can be leveraged to develop best practices for production of accessible EPUB 3.
· The results are highly dependent on the contributions of DAISY Community as well as inclusive publishing community.
· Project has dependency on the collaboration with the W3C
· Emergence of similar kind of evaluation methodology with stronger political backing or market forces.
· There is a possibility of criticism if some reading systems developers do not like a 3rd party reviewing their products.