{"id":1415,"date":"2014-11-18T06:22:10","date_gmt":"2014-11-17T21:22:10","guid":{"rendered":"http:\/\/www.lookingatnothing.com\/?p=1415"},"modified":"2014-11-28T16:52:15","modified_gmt":"2014-11-28T07:52:15","slug":"looking-at-nothing-seeing-a-lot","status":"publish","type":"post","link":"https:\/\/lookingatnothing.com\/index.php\/archives\/1415","title":{"rendered":"Looking at Nothing, Seeing a Lot."},"content":{"rendered":"

Today is a day of relief for Dr. Julian Stirling<\/a> and his eight co-authors (with many looking forward to the response, including Rapha\u00ebl L\u00e9vy<\/a>). The paper released today<\/a> opposes ten years of prolific work from a group claiming to have made and observed stripes on the surface of nanoparticles (c.f. Figure 0, Figure 1 in this post). While most of the work revolves around scanning probe microscopy (SPM), small-angle scattering also played a minor role (c.f. Figure 2 and this paper<\/a>). This, coupled with modern approaches to publication, led to my inclusion in the (otherwise amazing) list of authors. Here is how this came to be.<\/p>\n

<\/p>\n

Despite it being a core principle of science, critical assessment of published work remain disproportionately rare<\/a>. While there are plenty of examples from the past (some more critical than others) [1970<\/a>, 1976<\/a>, 1986<\/a>, 1996<\/a>, 2004<\/a>,\u00a02007<\/a>, 2007<\/a>, 2011<\/a>, 2014<\/a>], it is obviously a tough endeavour with little reward. The toughness was illustrated by Alberto Brandolini, who said that: “the amount of energy necessary to refute bullshit is an order of magnitude bigger than to produce it” [source<\/a>, source<\/a>]. Critical assessments, then, are works of immense effort (it is science on steroids).<\/p>\n

There are more practical and social factors at play besides the challenges of investigating and writing critical papers. Socially, it appears to have become less accepted to criticize someone else’s work (with professional criticism of work being equated to cyberbullying<\/a> by Dr. Stellacci (more on that here from Dr. Moriarty<\/a>)). Furthermore, it is often not considered to be a “cool” part of science to assess, pick apart or replicate someone else’s work. It can also lead to lifetime feuds, this particular critical work<\/a>, for example, is said to have ignited a life-long ice age between the authors and those whose work was assessed (acerbic comments aplenty in subsequent papers. Dr. Ruland and coworkers fortunately never stopped being critical: here<\/a> is one of Dr. Burger and Dr. Ruland from 2006).<\/p>\n

Practical problems also arise when “high-impact” journals like Nature Materials consider it uninteresting, or even outright refuse to consider reassessments of existing data (not very scientific of them, to say the least) [example<\/a>]. The increased pressures to publish a lot of small papers quickly also runs counter to the aforementioned larger effort required, especially when evidenced by Dr. Cesbron and Dr. L\u00e9vy’s three year struggle<\/a> to publish his prior critical work on the stripy nanoparticles (a writeup about that here<\/a>, and the frustrating timeline here<\/a>).<\/p>\n

A new, more comprehensive re-assessment of the stripy data was subsequently written by Dr. Stirling. For this work<\/a>, some of the original evidence for stripes was scrutinized, and suspicions raised in the previous work were substantiated: some of the early work clearly suffers from feedback loop artifacts. To drive home the point, similar features are both generated in simulated as well as practical SPM measurements.<\/p>\n

\"Figure<\/a>
Figure 0:<\/strong> Figure from Ong et al. 2014<\/em>, one ostensibly showing stripes, the other not (see if you can tell).
Reproduced from DOI: 10.1039\/C4CC04114C <\/a>with permission requested from The Royal Society of Chemistry on Saturday Nov. 15, 2014. Permission granted Nov. 28, 2014.<\/figcaption><\/figure>\n

The later stripy nanoparticle publications showed drastically less pronounced features than the first flawed results. The better the instruments, the lower the visibility of the stripes. Ostensibly stripy features are brought out in these later images through drawing of suggestive arrows, lines and other marks through otherwise unremarkable features (harking back to those funny optical illusion tricks of high school). See, for example Figure 0, and compare with the imagined stripy nanoparticle morphology shown in Figure 1 (both published in 2014).<\/p>\n

At last, several of the last papers showed evidence from techniques other than SPM (although it is astounding that supporting evidence has not been required for any of the other papers in high “impact” journals!). We see arguments based on data from FTIR (which cannot distinguish stripes), TEM (showing nothing), nuclear magnetic resonance (NMR), and small-angle neutron scattering (SANS) that are supposed to provide independent evidence.<\/p>\n

\"Stripy<\/a>
Figure 1: <\/strong> Stripy nanoparticles are supposed to look like this. Adapted with permission from Huang et al. 2014. Source DOI: 10.1021\/nn501203k<\/a>
Copyright 2014 American Chemical Society.<\/figcaption><\/figure>\n

Fortunately, an NMR expert was already available for the first manuscript in the form of Dr. Josef Granwehr<\/a> to address these. In short: evidence for phase separation on the surface, but no evidence for stripy morphologies.<\/p>\n

The authors then wrote a small section in their paper discussing the then very recent small-angle scattering evidence (published only very shortly before the final draft was submitted to PLOS One<\/a> and arXiv<\/a>). While their initial assessment of the SANS results were reasonable, there were several aspects that could be improved.<\/p>\n

As the authors were of the modern variety, they published their draft on the arXiv preprint server and tweeted about it. This meant that I became aware of it, read the manuscript, and wrote about it on this blog<\/a> (and a bit more with video<\/a> later). I also contacted the authors of the paper in January with suggested improvements. To my (happy) surprise, they were suggesting adding me on as an author a month later after some more e-mail discussions.<\/p>\n

\"STM<\/a>
Figure 2:<\/strong> STM image of “stripy nanoparticles” and structure retrieved from SANS data.
Reproduced from DOI: 10.1039\/C3SC52595C<\/a> with permission requested from The Royal Society of Chemistry on November 15, 2014.<\/figcaption><\/figure>\n

At the same time, a massive discussion<\/a> was raging at the PubPeer forums on the submitted manuscript. Participating in this discussion was very interesting, as it proved a platform for training argumentative skills as the debate heated up (highlighting once more that we do not get the necessary but important training on argumentation at university, but this<\/a> helped me quite a bit (a PDF of that book can be found by googling the title and author)).<\/p>\n

With the refinements based on the PubPeer discussion and (some) reviewer comments integrated in the paper, the manuscript was much improved and sent out for review again. Here<\/a> are some of the reviewer comments. My paragraphs can be read at the very end of the paper (in short: no evidence for stripy features, and (moreover) circumstantial evidence against<\/em> stripy features).<\/p>\n

What followed was a frustrating time in which the PLoS One editor became unresponsive, and a host of other issues including copyright on the criticized images (an issue highlighted by Julian Stirling here<\/a>, and by PLOS One here<\/a>). After kicking up more fuss on social media (to which there is surprising sensitivity), this too was resolved.<\/p>\n

I really cannot do justice to the full timeline in this post. All I can say is that it is very similar to that experienced<\/a> by Dr. Cesbron and Dr. L\u00e9vy. Given the essential position of critical assessments in science, it surprises me that there really is no easier way to publish such considerations. We are fortunately seeing increased ease of critical thinking through PubPeer, but that can not, and should not be the final venue. Once criticism has crystallized to brief, intelligible points, it should be possible to publish the summary as any paper, and do so quickly.<\/p>\n

N. B.: Since the first draft of this manuscript became available in December 2013, there have been another seven publications on stripy nanoparticles by Dr. Stellacci and coworkers. Three are directly about characterization of stripy nanoparticles (including the SANS results)[Chem. Comm.<\/a>, Chem. Sci.<\/a>, Israel J. Chem.<\/a>], and four are utilizing or building upon the idea of the stripy nanoparticles without evidence of their existence [ACS Nano<\/a>, Nature Comms.<\/a> ACS Nano<\/a>, Nanoscale<\/a>].<\/p>\n","protected":false},"excerpt":{"rendered":"

Today is a day of relief for Dr. Julian Stirling and his eight co-authors (with many looking forward to the response, including Rapha\u00ebl L\u00e9vy). The […]<\/a><\/p>\n<\/div>","protected":false},"author":2,"featured_media":1425,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-1415","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/lookingatnothing.com\/wp-content\/uploads\/2014\/11\/GA.gif","jetpack_shortlink":"https:\/\/wp.me\/p1gZ2v-mP","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/posts\/1415","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/comments?post=1415"}],"version-history":[{"count":16,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/posts\/1415\/revisions"}],"predecessor-version":[{"id":1456,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/posts\/1415\/revisions\/1456"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/media\/1425"}],"wp:attachment":[{"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/media?parent=1415"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/categories?post=1415"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lookingatnothing.com\/index.php\/wp-json\/wp\/v2\/tags?post=1415"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}