How optimism about buggy high throughput data might again cause damage (wonkish)

I just mentioned an unexpected negative consequence of scientific optimism about new methods– an instance in which optimism led scientists to race to generate, journals race to publish, and companies to race to capitalize buggy data, and how those developments precluded later generation of better data.

I raised this example because one aspect of 2016 is a wave of seemingly starry eyed optimism about CRISPR / Cas9. This is a case where a technical improvement really is huge and has already sparked a wave of subsequent technical invention that is far from peaking. At such times, we might be tempted to imagine that the technology will solve every problem.

One of the things it’s overall not going to solve, at least in the way now being hyped, is reporter genes.

As Doudna and Charpentier envisioned, CRISPR/ Cas9 provides unparalleled ability to fuse coding sequences of genes to useful tags, including affinity purification tags (Savic et al. 2014), degradation tags (eg. Park et al. 2014), and fluorescent protein (XFP) tags (Paix et al, 2014). With strains and cell lines containing XFP tagged proteins in hand, researchers can quantify protein measuring fluorescence at the wavelength emitted by the XFP.

Since the 1960s, scientists have, by various means, including means that predate recombinant DNA itself, fused different promoters to the coding sequences of genes whose protein products are easily quantified. constructed reporter genes.   Starting with the early constructions in the bacterial chromosome or on live or defective bacteriophages such as the φ80 trp-lac fusions in E. coli (Miller et al.1970), scientists have used expression from reporters in qualitative genetic experiments to understand how mRNA synthesis from the fused promoter is regulated and to isolate enlightening mutants.  A lot of biological experimentation still depends on reporter genes.

As with the use of large scale protein interaction in the ’90s and ’00s to mobilize investment capital, I wouldn’t be surprised if some of the attention to using wholesale gene fusion via CRISPR/Cas9 to undertake comprehensive surveys of gene expression wasn’t due to this particular application being driven, somewhere, by a desire to raise money for some company.

But even far from Sand Hill Road, the same people who are agog with the possibility of whole-genome protein expression tagging by CRISPR/ Cas9 may not have considered how important it is to get reporter genes whose expression very precisely mirrors that of the promoter they are studying

This whole-genome stock-price-related issues is on my mind because my lab is trying to uses reporter genes in one particular limit application, to help sort out that part of variation in gene expression that is due to differences in signal from different signaling systems that reaches various promoters. To measure this variation, we need to minimize other sources of experimental error that we might falsely interpret as meaningful variation.  The way to handle this is for the constructions to be identical in all but the differences among those promoters. Operationally, this means same protein, same sequences governing ribosome binding, same 3′ UTR, 5′ UTR as similar as one can get it, same number of gene copies, and– this is an important part– all assayed genes parked in otherwise-genetically-identical organisms at the same chromosomal sites.  Otherwise we know (Pesce et al. 2016) that the amount of variation in reporter expression can be impacted, for example by colliding RNA polymerases coming in from the opposite strand (Saeki and Svejstrup 2009) increasing gene expression noise).

Put more simply, to understand of the causes and consequences of variation in gene expression, that we need to find, control for, and eventually extirpate confounding contributions measurement of this variation.

And yet, a prediction. There will be a lot of gene expression data generated from collections of CRISPR/Cas9-generated fusion proteins that fuse fluorescent reporter genes to different coding sequences. These proteins will be expressed from sites scattered all over the relevant genomes. The promoters will be different, as will 5′ untranslated mRNAs, the 3′ UTRs, and the protein coding sequences. mRNAs and proteins will be transcribed, translated and degraded at different rates. There will be variation in measured signal to promoter due to other polymerase II transcription complexes blundering through the same coding sequences. Studies using hese gene fusions studies will be hyped by the people who carry them out, and hyped by the journals that publish them. Results will include important qualitative conclusions. Results will also contain quantitative data, which will be used by certain kinds of computational biologists to generate many hundreds or even thousands of publications that few will further appreciate. And, as with protein interactions, once published, these badly controlled gene fusion studies will make it more difficult to carry out more careful work.

I don’t see a moral to this, except maybe the old one that the pursuit of the new, the shiny, the purportedly transformative sometimes acts to make more difficult the pursuit of longer term questions. Scholars were probably complaining about the pursuit of the shiny and its distorting effects in the 800s (Francoise 1996, Al-Khalili 2011). So that part is not historically unique.

On the other hand, as we saw last year in the rise and fall of the myth-fullfiling, TED-talking, Lisa Holmes (Carreyrou, 2015), the robustness of the contemporary connection between a good transformative story and the ability to access capital to start companies is historically unique.

But this is the institutional landscape in which we now do our science.

References

Al-Khalili, J.  (2011) The House of Wisdom: How Arabic Science Saved Ancient Knowledge and Gave Us the Renaissance, New York: Penguin Press, ISBN 9781594202797

Carreyrou, J. (2015) Hot startup Theranos has struggled with its blood-test technology.  Wall Street Journal, 15 October 2015.

Francoise, M.  The Scientific Institutions in the Medieval Near East, pp. 985–1007 in Rashed, Roshdi; Morelon, Régis (1996). Encyclopedia of the History of Arabic Science: v.3 Technology, alchemy and life sciences. Routledge. ISBN 9780415020633.

Miller, J. H., W. S. Reznikoff, A. E. Silverstone, K. Ippen, E. R. Signer and J. R. Beckwith (1970) Fusions of the lac and trp Regions of the Escherichia coliChromosome. J Bacteriol 104: 1273-1279.   PMID: 16559103

Paix A, Wang Y, Smith HE, Lee CY, Calidas D, Lu T, Smith J, Schmidt H, Krause MW, Seydoux G.  (2014).  Scalable and versatile genome editing using linear DNAs with microhomology to Cas9 Sites in Caenorhabditis elegans.  Genetics. 198(4):1347-1356. doi: PMID: 2524945

Park A, Won ST, Pentecost M, Bartkowski W, Lee B (2014).  CRISPR/Cas9 allows efficient and complete knock-in of a destabilization domain-tagged essential protein in a human cell line, allowing rapid knockdown of protein function.  PLoS One. 2014 Apr 17;9(4):e95101.  PMID: 24743236

Saeki H, Svejstrup JQ (2009).  Stability, flexibility, and dynamic interactions of colliding RNA polymerase II elongation complexes.  Mol Cell. 35(2):191-205.

Savic D, Partridge EC, Newberry KM, Smith SB, Meadows SK, Roberts BS, Mackiewicz M, Mendenhall EM, Myers RM.  (2015).  CETCh-seq: CRISPR epitope tagging ChIP-seq of DNA-binding proteins.  Genome Res. 2015 Oct;25(10):1581-1589. PMID: 26355004

Pesce Pesce, C. G., Zdraljevic, S., Peria, W., Rockwell, D., Yu, R. C., Colman-Lerner, A., and Brent, R. Cell-to-cell variability in the yeast pheromone response: high throughput screen identifies genes with different effects on transmitted signal and response.  Still unpublished.