When someone is editing a LEMDO text, they’re making use of two different schemas, which Oxygen is aware of in the background; these include a RELAX NG schema (lemdo.rng) and a Schematron schema (lemdo.sch). If you’re a programmer creating e.g. XSLT code for the project, there is another schema which helps to standardize the codebase: technical.sch. Finally, when the various TEI versions of each XML file are created at build time, they are also validated against a second pair of schemas, tei_all_LEMDO.rng and tei_all_LEMDO.sch.

All of these schemas are found in the data/sch folder, alongside the files from which they are generated: lemdo.odd, technical.odd, and tei_all_LEMDO.odd. You will also see three matching XSLT files, lemdo.xsl, technical.xsl, and tei_all_LEMDO.xsl. These are the pre-processed versions of the Schematron (*.sch) files used to do Schematron validation. The Schematron language is transformed into XSLT before being applied to a document for validation purposes, and these XSLT files are pre-generated so that this process only has to be done once.

Schemas and their associated sources are only usually edited by senior editors, project managers, or programmers. Anyone else who would like to see a change to the schema should make a suggestion to the project leaders, or raise an issue on GitHub.

Whenever a change is made to a LEMDO schema, it is usually done in the appropriate ODD file (although not always: see below). If, for example, a new Schematron rule is added (in the form of a TEI <constraintSpec> to the lemdo.odd file, the schemas will have to be rebuilt before that rule can take effect. The schema build process is handled by an Ant file, code/schema/buildBaseOdd.xml. This file builds all three ODD files to generate all their outputs, and handles all complexity automatically. We recommend that you run this every time you make any changes to any of the ODD files or related files, even if it seems that some rebuilds may not be required. It doesn’t take long to run.

To build the schemas from the ODD files, you need to be on a Linux or Mac computer; Windows will not work. You can run the build process in two different ways:

The results should be exactly the same: all the associated files will be refreshed, and changes can be committed if they are as expected. Before committing changes, though, we recommend that you validate all the existing files as follows: Navigate to the lemdo root folder, and run:
svn update && ant quickValidateSource
This will take some time to run, but it will validate all the existing XML files in the project with the rebuilt schemas. If the change you made has caused any of these files to become invalid, you will need to figure out what the problem is.

The lemdo.odd file in particular is large and complicated, and if you look through it, you will see some sections that have comments like this:

These comments appear in locations where the ODD content is not constructed by hand, but is generated as part of the schema build process, taking XML structures from the TAXO1.xml document (typically taxonomies/categories) and generating from them (for example) <valList> elements in the ODD file, which are then processed into attribute values in the RELAX NG schema. We do things this way because it’s easier for senior editors to maintain taxonomies in the form of TEI <taxonomy> elements than to hand-edit the ODD file elements, but also because we use the <taxonomy> elements in other ways, such as rendering them into documentation tables. A <taxonomy> ’s descendant <category> elements can be usefully organized into a nested hierarchy, whereas a <valList> has to be flat.

This means that when you need to make a change to some component in the RELAX NG file, you should first check to see if it is encoded directly in the ODD file, or whether the content in the ODD file has a comment telling you that it is generated from somewhere else; if the latter is true, you will need to make the change in the other location before regenerating the schemas.

In the lemdo root directory you will see a directory called Stylesheets-local. This contains a local copy of the parts of the TEI Stylesheets collection which are required to build ODD files into schemas, along with a file called p5subset.xml, which is a complete copy of all the TEI element and attribute definitions from the TEI standard. We use static copies of these files rather than reaching out to the TEI website to get the latest fresh copies of them for two reasons:

Generally, we like to stay up to date with the latest versions of the TEI and Stylesheets, but because updates may result in problems, we need to stage them carefully. When a new version of TEI is released, we recommend that a programmer on the project approach the update as follows:

If everything succeeded—in other words, if the schemas all built successfully, and all the LEMDO documents were valid—then you can commit all the changes. However, if anything did not work, you will need to figure out and solve the problem, or roll back all the changes:
svn revert --recursive .

The LEMDO project guarantees that all previous editions of an anthology will remain available permanently, so that any existing citation pointing to an older anthology can still be verified. For this reason, we do not overwrite previous editions with new editions. Instead, they exist side by side. Each anthology has a unique lower case identifier (douai, classroom, moms, peer-review etc.) which is used in build commands, and is also the name of the directory in data/anthologies which contains the anthology.

Editions are published at the URL: https://lemdo.uvic.ca/[anthology]_editions/[edition_number]. So, for example, you will see:

Alongside the _editions folder is a symlink (symbolic link) which is named for the edition (here “classroom”), and which points to the latest edition:

classroom -> classroom_editions/0.2

So anyone who browses https://lemdo.uvic.ca/classroom will be looking at the latest edition; but anyone who wants to see a specific edition can browse the url for that edition.

The anthology template files all include an XHTML <h2> element which appears as the first item in the <article> element. That <h2> is the trigger leading to the insertion of the page title(s).

We never take the page title from the content of the TEI <text> element. Our basic assumption is that the title appearing at the top of a page on the site is drawn from the content of the <titleStmt> in the <teiHeader> , according to these rules:

<front> elements should only be used in primary source documents, and should only contain genuine transcribed front matter from a primary source document. Any front element in a primary source document will be processed and rendered following the main page title created following the rules above.

If you need to provide bylines, prefatory statements, or other such content at the head of the page in a born-digital or mixed-content document, put this information at the beginning of the <body> .

LEMDO has thousands of <ref> elements that point to editions, anthologies, components therein, and to resources outside LEMDO. These are all processed into HTML hyperlinks for the static site.

Our processing does not do anything with <ref> elements in PDFs. Any pointing you do inside a file with the <ref> element will result in nothing in a PDF.

The <ref> and <ptr> elements can co-exist because they are pointing to xml:ids, but only the <ptr> element can be converted to strings at build time in LaTeX.

Example: Processing of a Reference in a PDF

We would like to have a canonically-structured textual reference in the output. In the digital edition, we want the A.S.Sp. system (e.g., 5.1.2) plus a precise hyperlink. In the print edition, we want the A.S.L reference system (e.g., 2.3.101). We do not want to have the author of the critical text write a literal 2.3.101 into their document, because lineation may change as the text is edited, but we do want a critical introduction to be able to contain A.S.L citations when it is printed. The actual text in the parenthetical citation must be generated at build time.

LEMDO therefore has two different processing chains for pointers: one for the digital edition and one for the print edition (a camera-ready PDF that can be downloaded or printed through print-on-demand services).

For online publication, we generate a parenthetical citation that gives LEMDO’s canonical reference system (A.S.Sp.). Clicking on the citation takes one directly to the part of the speech being cited.

For the PDF, we generate a parenthetical citation that gives A.S.L (act, scene, line number) using the line numbers generated for the PDF at the time we make the PDF.

For example, an editor might use a <ptr> element in their critical introduction to point to anchors in the middle of a long speech in their modernized text. In the processing for the PDF, LEMDO will calculate and supply the A.S.L, so that reader may find the exact line(s) being cited in the generated parenthetical citation. For the online version, the parenthetical citation will be A.S.Sp. but the hyperlink on the citation will go directly to the target point in the speech.

If editors are writing a critical introduction to Othello, they will naturally want to provide links to other relevant texts in the LEMDO collection. This is normally done using the <ref> element, like this: This points to the modern edition of Titus by its document id. However, if you want to point to a specific location in that text, things become more complicated, in the sense that although you may point at an ID in that text, it’s not clear what textual reference you should provide. For example, if you call it 2.5.35 (Act 2, Scene 5, Line 35), and then the editor of Titus determines that there are two missing lines at the beginning of the scene, and adds them in, the text inside your <ref> element is now misleading. The following sections provide two solutions to this problem, one used within a specific edition directory , and one used more generally.

One approach to the mutability of online texts produced both within and outside LEMDO is to choose a specific print edition of source texts (such as the New Oxford Shakespeare) to which all references can point. This has the obvious disadvantage that such a pointer cannot be made into any kind of link, but links are fragile anyway, and this approach also fits with the numerous citations of critical print texts which occur throughout critical material.

We have seen above that certain types of citation between texts are not very robust, because texts are (for the forseeable future, anyway) steadily evolving, and even our principles for lineation and our citation styles are not absolutely finalized. However, when you’re editing critical material that will be bundled with an edition of a text in (for example) a print volume, you need to be able to point into the text, just as you need to be able to attach annotations to specific points in the text. There are two scenarios in which we do this:

Pinning Annotations

Local Pointers

LEMDO prefers the use of processing instructions for the purposes of inclusion over other methods such as XInclude because it is more flexible; processing for XPointers in XInclude instructions is not widely supported, and some processors and validators may act upon XInclude instructions when they’re not intended to be processed. There are two PI-based inclusions in LEMDO:

lemdo-include

<?lemdo-include href="doc:learn_encodeLinks_intro"?>

lemdo-import

<?lemdo-import ref="doc:emdCAMP3_edition"?>

Another set of processing instructions provides a way to generate content in an output page based on metadata or data elsewhere in the project. These are three examples:

category-listing

<?category-listing ref="emeeTheatre"?>

taxonomy-table

<?taxonomy-table ref="emdAudiences"?>

charDecl-table

<?charDecl-table ref="characters"?>

listPrefixDef-table

<?listPrefixDef-table ref="global_listPrefixDef"?>

roleList

<?roleList ref="emd1HW_M"?>

msDesc

<?msDesc ref="emdDouai_AYL"?>

LEMDO sites are static sites conformant with Endings principles, so they use the Endings-developed staticSearch system to provide search functionality for individual anthologies. Rather than check out a fresh copy of the staticSearch codebase every time we build something, we store a static copy of the codebase in our repository in code/search/staticSearch. This should be an up-to-date stable version of the codebase.

When any anthology is built, the staticSearch codebase is copied into a staticSearch folder in the anthology’s products directory. We could run all our staticSearch build indexing for all anthologies directly from the code/search/staticSearch folder, but making a copy enables us to do tests with alternative versions of staticSearch if we need to, using a single anthology.

For details on how staticSearch itself works, see the staticSearch documentation.

The file build_globals_module.xml contains a property called staticSearchBranch which specifies which branch we want to use for our staticSearch codebase copy. It should normally be set to a stable release branch, unless we are doing some unusual testing. Release branches are updated periodically, for bugfixes and minor enhancements, so there is also an Ant task in the same file called refreshStaticSearchCode, which will update the files in the code/search/staticSearch folder automatically. After running the update (ant -f build_globals_module.xml refreshStaticSearchCode), check the svn status of the code/search/staticSearch folder to see whether there are any new files that need to be added, or perhaps files that need to be deleted.

The encoder documentation provides good info on how witnesses should be encoded. A <listWit> appears in the <sourceDesc> of a collation file, accompanying the apparatus list which is encoded in the body of the document. Each <witness> element represents a single source text which was used by the editor in creating the collation.

A Schematron rule constrains the <witness> element such that it either has @corresp and is itself empty, or has a tagged prose description of the witness and does not have @corresp. The first scenario is used when the BIBL1 entry pointed at by @corresp provides sufficient information and no further explanation is needed. In the second case, the editor provides a prose description which is more elaborate, but is expected to link some of the contents of that description to one or more entries in BIBL1.

Witness lists are part of collations, and stored in collation files, but the build process that creates a fully-realized text combines the text of the work with the collations to produce an integrated whole. During the generation of standalone XML, the <listWit> is first copied into the play source file (always a modern edition). At that point, if a <witness> element is empty and it has @corresp, the target bibl element is copied into the content of the witness (minus its @xml:id), and the @corresp attribute is removed.

At the HTML stage, the witness list is processed into a sequence of <div> elements in the appendix of the document, along with the bibliography items, person elements and so on. These elements are hidden by CSS and shown only in response to user actions such as clicking on a linked name. Apparatus elements are also part of the appendix. When a user clicks on a collation icon, the relevant apparatus item appears in a popup. In that popup, each collation siglum is a link to its witness element, and clicking on that link causes the JavaScript to retrieve the witness info and append it to the bottom of the popup. Thus the detailed witness info is always available from any apparatus popup.

The print editions are generated using LaTeX, and specifically the Xelatex compiler, so anyone wanting to generate a print edition will need to have not only Ant and ant-contrib but also the (substantial) LaTeX codebase installed. On Linux, we recommend installing the texlive-full package, which should include everything you need. On Mac, you can use the mactex distro. You can also install texlive on Windows, but we do not expect most of our build processes to work on Windows for a variety of reasons. *NIX-based OSes are a much better bet. The distributions are large, so don’t install this stuff just for fun; only do it if you have a need to build PDFs for print.

The PDF build file includes a quick check for the availability of the Xelatex compiler, so starting from the repository root, you can do this: If this works, you’re probably OK, although it’s always possible that a particular package required by the build process is not installed. If that’s the case, when you try to run a build, you should see helpful error messages from LaTeX.

All other requirements (specifically, the fonts used to build the PDF) should be in the repository.

As you might expect, the codebase for building a print edition lives in code/pdf. It is basically very simple:

Once you have ensured that your system is set up with all the requirements, and you have set up your TEI print edition document, you’re ready to try a build. Starting from the LEMDO repository root, this is what you do: (You can supply multiple IDs, comma-separated, if you want to.)

The build process will create a folder called pdf inside the main folder of the text you are building. In there, a number of files will be saved, including a log file, the .tex file containing the LaTeX code which is generated and then compiled, and the PDF file of the print edition. If anything goes wrong, you should see either helpful messages from our code or mysterious messages from the LaTeX compiler.

During the build process you will see many very puzzling emanations such as the common Underfull \hbox (badness 1033) message from the compiler. These are mostly informational, warning you when the layout engine has had to stretch or squash a line a little more than it would like to in order to get the justification to work. However, if the build actually fails, you will need to pay attention to whatever message coincides with the termination of the build.

You will notice that the Xelatex compiler is run at least four times on the .tex file. This is because at each run, the layout engine is able to do a slightly better job of adjusting spacing, pagination and so on, but every time it does this, page references and similar content which were generated at the time of the previous build are potentially no longer accurate, so another run is recommended. The number of runs required to get a final version is not easy to determine, so we run four times by default, but this may need to be adjusted.

A LEMDO print edition is established by creating a standard LEMDO TEI file, but with a filename ending in _Print.xml. This should be created in the main folder of the work itself. So for example, if you’re creating a print edition of Othello, you would create this file: data/texts/Oth/main/emdOth_Print.xml

This file is like any other TEI text in the project; it has metadata, responsibility statements and so on. But it will mostly consist of content from other parts of the edition directory. Primarily, it will include a modern-spelling edition of the play, but it will also have other components such as critical materials and a bibliography. The following is a simplified example which will be explained below, and should cover all the main components.

The header is a normal header except for the particular document type specified with <catRef> /@target="cat:ldtBornDigPrint".

But in <text> , the first thing you will notice is that the <front> , <body> and <back> elements do not include any content directly. They can include content if necessary, and there may well be components that are intended to be used only for one particular print edition, and therefore belong directly in this file, but most content is in the form of <linkGrp> elements containing pointers. These pointers specify other files in the repository, or sections of files. They use the doc: prefix to point to the ids of files, and an optional fragment identifier to point to a specific part of the file. These includes will be processed by the first stage of the build code to create a complete TEI file incorporating all these components. That structure will then be processed into the TEI file.

Notice the organization: critical materials come in the <front> element, the castlist and the play itself come in the <body> , and the bibliography (of which more below) appears in the <back> . Also in the <back> are any annotation files which are needed; these are processed into footnotes at build time. Only annotations which are actually referenced in the included texts will be used, and the rest will be discarded.

Finally, note the special case of the <divGen> element for the bibliography. This is acted on by the build code, which retrieves from BIBL1.xml all bibliography items which are actually mentioned in the rest of the content, and generates a bibliography from them automatically. Note that if there is a reference to an item which does not appear in BIBL1.xml, the PDF build will fail and stop.

A print edition is normally built from regular edition files and supplementary materials, but there are a few cases in which special encoding may be required. Some of these are covered in Remediate Annotations for Print, Encode Annotations, and Types of Annotations.

One feature of print editions is the Table of Contents which is automatically generated at build time from the contents of the combined print document. This is constructed from all the <head> elements that appear in the text. However, sometimes it is desirable not to include a particular heading as an entry in the table of contents (where, for example, a short section or chapter has a lot of headings which would all be listed as being on the same page). In this case, you can add @type to the <head> element as in this example: If you do this, the <head> element will be ignored when creating the Table of Contents for the print volume.

This file explains the relationship between documentation files and the project’s ODD file. It provides a quick and general overview of the files involved in the corresponding processes, and then describes the processes in general, without too much attention to the technical details.

Below is a list of code files stored in lemdo/code/site/xsl/documentation, responsible for generating and processing documentation files, along with their role in the process.

We write our documentation in files that are rooted on <div> elements and we store them in the lemdo/data/documentation folder. These files are then included in the ODD file via processing instructions and a series of XSL transformations. Every chapter is a <div> in the body of the ODD file, and every <div> includes a series of processing instructions that are later processed into the sections (pages) of a certain chapter. The sections are the files from data/documentation. The chapters are generated programmatically, as instructed per the ODD file. This process happens through the XSL files as described above.

The documentation_to_documentation_chapters_master.xsl file is also responsible for creating the documentation index, from the chapter <div> s as they are laid out in the lemdo.lite.odd stage of the documentation creation process. While the chapter titles (Chapter 1. Quickstart Guidelines, etc.) are determined in the odd file, the section headers are determined in the <head> child element of the root <div> element in the corresponding documentation file.

Please see for detailed guidelines. We write the documentation with a root <div> element so that we can append the file content to the LEMDO ODD file as we generate it. From the ODD file we then grab corresponding <div> s and create the proper structure for them, adding header and body elements. This process begins with the documentation_inclusion_master.xsl, which includes the <div> elements in the LEMDO ODD file, as instructed by a series of processing instructions. The documentation_to_documentation_chapters_master.xsl and documentation_to_documentation_sections_master.xsl files process the documentation <div> elements from the LEMDO TEI Lite ODD file and make them into their own standalone files, in XML as well as HTML.

All of the above happens automatically as part of the build process. However, you may want to test your documentation locally, in XML or HTML. To build XML files, go to code/build_documentation_master.xsl, and press the play button. The files are outputted in lemdo/chapters_out and lemdo/sections_out. To locally build your documentation files into html files, open Terminal and type the following command: ant createDocumentation. You can find more on local builds in Run Local Builds.

In LEMDO, element documentation is generated directly as imported from the TEI P5 Guidelines. Given the nature of the project and the specialized uses for TEI elements, LEMDO adds notes and modifies TEI abstracts where necessary. This process is accomplished in the <elementSpec> elements of the corresponding element in the ODD file. To add a note to the existing element structure and descriptions, follow these steps: Note that the <gloss> and <desc> elements and their original TEI content are not present in the ODD file, they are automatically imported from the TEI guidelines at processing time. If you would like to add a note to the existing TEI descriptions, follow these steps: Your note will appear in the table of specs on the element’s page on the LEMDO site, with the label Note.