A rant about data quality: machines first, humans second…

GraphicalAbstractRecently one of my papers emerged through the publication system of Journal of Cheminformatics, entitled “Machines first, humans second: on the importance of algorithmic interpretation of open chemistry data“, co-authored with Antony Williams and Sean Ekins, and incorporated into the JC Bradley Memorial Issue. Spoiler alert: the paper is about how if you’re publishing open lab notebook data without adhering to rigorously defined standards for machine readability, then you’re mostly wasting your time, and arguably making the open data situation even worse than it already is. The tone of the article is a bit less polite than I normally try to be, so fair warning, but it’s all for a good cause.

1 Comment

XMDS: progress toward structure sketcher

xmds_sk01Since the last sneak preview, the skunkworks project “XMDS” – the Mac OS X desktop version of the Mobile Molecular DataSheet app – has gained enough functionality to make another screenshot, this time showing what the actual molecular drawing interface might look like once it’s done. Read the rest of this entry »

1 Comment

Mobile Bayesian modelling: importing

importbayesOne more key piece is in place for Bayesian modelling with apps: MMDS 1.6.1 just got approved on the AppStore, and brings with it the ability to recognise files with the .bayesian extension (or MIME type chemical/x-bayesian), and import them into the collection of available models that can be used to calculate properties. At the present time the only official way to create such models is to use the bleeding edge build of the Chemical Development Kit and roll your own wrapper code, but we’re working on that!

Leave a comment

Recent developments with Bayesian models and app data sharing

bayes_recent1Several of the flagship apps from Molecular Materials Informatics have had major updates recently: the Mobile Molecular DataSheet, SAR Table, MolPrime+, Green Lab Notebook and Approved Drugs. Two separate groups of features have motivated these updates: (1) the inclusion of in-app calculation of nontrivial properties, lately supplemented by the inclusion of Bayesian models, and (2) leveraging the new iOS 8 API feature for importing & exporting data to any compatible service, which includes iCloud by default, but also Dropbox if it is installed. Read the rest of this entry »

Leave a comment

Dabbling with the desktop: molecular datasheet app on the mac

xmds_firstpreviewThis is a way-too-soon sneak preview of what might someday end up as a commercial product. Currently residing under the project acronym XMDS, it is related to the MMDS app, for Mac OS X (i.e. X-Molecular DataSheet). The minimum viable feature set is intended to be approximately equivalent to the datasheet & molecule editor that is implemented by the open source SketchEl project.

The screenshot above shows the sum total of the functionality of the Mac app so far: the ability to load a datasheet file into a window, create an outline grid, and render each of the cells. Obviously the text and numbers are not too difficult to render, but as seen for the two structures, the first one is very simple and plausible, while the second one is nonsense. The latter exists for the purpose of verifying that a variety of different rendering scenarios are handled as intended. If you look carefully at the molecules, you may also notice that they are very carefully crafted, in order to demonstrate publication-worthy aesthetics from raw connection tables. The graphical rendering layout algorithm, which has the 2D coordinates of the heavy atoms (in Angstroms) as its only visual cue, is the same one that is used by the com.mmi software stack that drives the molsync.com services, and is described in detail in a paper published in Molecular Informatics. Even though the app currently does nothing other than display static content, the amount of machinery that had to be ported over is quite extensive, so this snapshot belies a significant amount of successful coding.

Internally, the project is interesting because it is all written using Apple’s not-really-ready Swift language. As I’ve previously written about the language (first and second impressions), this is a surprise announcement that Apple revealed in June 2014 after a secret incubation period, and for which the specifications made it seem that it was going to be a very fine programming environment right out of the gate, with some very clever and well thought out features. Unfortunately current reality is quite the opposite: the version number should probably be stated as “0.1 alpha”, and the development experience is correspondingly awkward, buggy and broken. Some of my early efforts to port cheminformatics algorithms to Swift (from their original Java implementations) revealed that for the kinds of low level algorithms that ought to perform excellently are actually something like 50-100x slower than a reasonably well written Objective-C version, which is a bit of a dealbreaker. The performance is more like what you would expect from naively implementing a number crunching algorithm in a scripting language. For this reason, I subsequently accepted the reality that my mobile apps for iOS would continue to be based in Objective-C language for a fair bit longer: it has been the victim of premature deprecation.

Nonetheless, since Apple has pretty much staked their reputation on their new language, it means they really stand to lose a lot if they don’t keep improving Swift so that it lives up to the hype in the reasonably near future. For what it’s worth I don’t think Apple is ready to flame out on its current winning streak, and so I’m betting that it is a good idea to start using Swift for products that may not be ready to release for awhile.

And hence the idea of expanding the product lineup of Molecular Materials Informatics to span a larger variety of platforms. While the most prominent software offerings are the mobile apps (iOS and Android), and behind the scenes with increasingly powerful serverside/non-interactive algorithms, the strategic plan has always been to cover the desktop platform, too. The lack of a clear winner in the platform/development space is one of the main reasons it is taking too long (i.e. Windows is big but past its prime, Linux continues to flail but is popular with several very important niche demographics, Mac keeps on winning but needs to hurry up and ditch Objective-C, cross platform tools like Java provide only the lowest common denominator user experience, the continued absence of a viable web runtime rules out browser options… and not to mention all the issues with deployment and installation). Swift promises to resolve the stalemate, and once the worst of its shortcomings are dealt with, it will win by default even if it is ultimately quite mediocre.

A large amount of the method development work that I do, in order to create cheminformatics algorithms for mobile apps and other platforms, requires me to personally edit molecular datasheets for many different purposes. When working on the desktop, I use the SketchEl open source package, which is written in Java. Recently I improved it so that it is a bit more Mac-friendly, since that’s increasingly becoming my preferred workstation choice, but I am frequently reminded of how much better it would be to have a native Mac replacement. The SketchEl project is extremely bare-bones in terms of algorithmic capacity: the datasheet editor is the absolute minimum necessary to make data entry and collection viewing reasonably straightforward for simple cases. Since the main functionality was finished up many years ago, the algorithms that are available within the proprietary codebases of Molecular Materials Informatics have a great deal of potential to augment it.

So don’t expect XMDS to appear in the Mac appstore in the next few weeks, but depending on a variety of different factors, it may see the light of day in the conceivably near future. Stranger things have happened.

Leave a comment

Online file sharing with iOS 8: upgrading the Green Lab Notebook app

picker_gln02Mobile apps for iOS have always been able to share files by a variety of different mechanisms, but many of these were limited in ways that were very detrimental to the user experience. The Green Lab Notebook app is now catching up to the new technology introduced with iOS 8: using the “document picker” interface to import and export files to document providers, which immediately makes it fully interoperable with iCloud, and file sharing services like Dropbox. Read the rest of this entry »

Leave a comment

Structure property calculation in apps: MMDS

mmds_propcalcs00An important milestone in has been reached in the migration of complicated structure-based calculations to pure mobile. The latest version of MMDS (1.5.9) is now available on the AppStore, and allows visualisation of calculated properties for individual molecules, as well as calculating new columns for entire datasheets.

The previous post described how recent porting of core technology (e.g. substructure query fragment searching) to Objective-C and iOS has opened the door to a variety of calculation types, including atom type-based contribution methods, while the post before that described how the porting of modern fingerprint types has enabled Bayesian models to be used. These progressions are significant, because the previous method of choice for carrying out difficult (or resource intensive) calculations was to hand off the data to a webservice, and await a response. The two technical arguments in favour of taking this approach are slowly but surely eroding: as device capabilities improve, the performance argument becomes less compelling, and as the difficult algorithms are migrated to Apple’s unique and incompatible-with-everything-else development tools, that stops being a problem as well.

The current version of the Mobile Molecular DataSheet has had two major features retrofitted: bringing up the calculation panel for an individual molecule now displays a wealth of calculated information, some of it in the form of numbers, some as graphically annotated structures. The calculation panel for whole datasheets offers the option for calculating scalar properties for each row, and storing the results in columns, which means that MMDS can be used as a calculation engine for other uses (e.g. QSAR or various kinds of visualisation). The individual molecule property calculation uses the same code as the MolPrime+ app, which received these new features first (but the second instalment is still awaiting review and should be available soon).

The properties that are now available and can be calculated locally on the app, with no internet connection or security conerns, include:

  • Easy-to-calculate scalar properties: molecular formula/weight, # heavy atoms, H-acceptors/donors, # rotatable bonds.
  • Log P & molar refractivity: both calculated by an atom contribution method (published by Crippen way back when), which requires implementation of substructure searching.
  • Bad valences: reviews the valence counts for each of the main group atoms and reports egregious mistakes (e.g. pentavalent carbon).
  • Stereochemistry: sites for R/S and E/Z stereochemistry are identified and their labels calculated, with unspecified or known ambiguous cases classified appropriately.
  • Tautomers: common H-shifts are identified and the complete list of tautomeric forms are enumerated, with duplicate equivalent molecules removed, and racemised stereocentres labelled accordingly.
  • PAINS filters: the original set of queries for identifying frequent hitters and other high throughput screening problem compounds is applied, and any matches are identified.
  • Mass distribution: the isotope distribution is calculated for integral masses, as well as the exact mass for the base peak.

The presentation of these calculated properties varies for single molecule vs. whole datasheet modes. Some numeric properties are displayed as such, alongside the structure:

mmds_propcalcs01 mmds_propcalcs02

Properties that are not inherently scalar are shown as structure overlays, for example valence mistakes which are identified in red:



Stereochemistry is also shown with the labels overlaid on top of the structure diagram:


However, it also induces a descriptive field, for counting the number of known/unknown stereocentres, and also the concept of “stereoambiguity”, which is essentially 2-N, where N is the number of unresolved stereocentres (so a compound with one unlabelled chiral or alkene-like stereocentre would have a value of 0.5, whereas a compound with two would be 0.25, etc.). This is the beginnings of an idea referred to as “confidence in chemistry”, which you are likely to be hearing more about soon from Chris Lipinski.

Tautomers are presented as an enumerated collection of compounds, when they apply:


Visually they are shown as a scrollable graphic, and in each case the atoms that are affected by a tautomer shift are highlighted. Any stereocentres that were distinctive in one of the tautomeric forms but have been the subject of one of the tautomer shifts are denoted as being ambiguous. For numeric purposes, a similar idea of “tautomer ambiguity” is calculated, with then confidence being 1/N, where N is the total number of tautomeric forms.

It should be noted at this point that some of these properties take some time to calculate. When the property viewing dialog is opened, it initiates a background thread, which grinds away at producing the results. Some of the properties are fast (e.g. molecular formula), some of them are just slow enough that they would glitch the user interface if they were not put in the background (e.g. log P requires a number of substructure matches), and some of them can take seconds, depending on molecule size and how new your device is, which in particular applies to the PAINS filters. These are built from a collection of SMARTS strings (upconverted to connection table queries, with the meta-sub-fragments expanded out, the total is close to a thousand) which all have to be run against the current molecule. The dialog panel updates as and when each of these becomes available:


As for tautomers, the PAINS matches are rendered for individual molecules as a side-scrolling collection. Most compounds hit zero-or-one of these filters, but it is possible to create molecules that hit many of them, and symmetry can crank it up further.

Calculating properties for a datasheet involves a less graphical setup dialog:


It consists of a checkbox for each of the properties that are desired. By default everything is on, except for the slowest calculation type (PAINS). Note also that the screenshot above shows an obscured section underneath, with the heading: Bayesian Models. This is the next major extension to the MMDS app, and it’s coming soon.




Get every new post delivered to your Inbox.

Join 1,167 other followers