Is "go big or go home" the way to go in academic publishing?

For the past few years, I have been struggling to reconcile my tendency to build large complete stories for publication and the need to show productivity for grants and promotion. I have agonized about what would be a manuscript that would satisfy both requirements, while I had to contend with deadlines, reviewer requests, delays, and personnel leaving. I am still not sure of what is the right way...or if even a right way exists, so I thought I'd brainstorm this here and see what people think.

I was trained to build substantial mechanistic stories so that a phenomenon would be reported with a mechanism attached to explain it. But these take time. I was also trained, maybe naively, to follow the most interesting question and identify whatever approach was most suited to answer it, which has led me to use an array of approaches and not to be technically specialized. This takes even more time. When you are in a large lab with massive resources in an institution with all kinds of expertizes you can draw from, this is greatly intellectually stimulating and a lot of fun. When you are in a small place where you are the only person doing what you are doing, and sometimes the only person in your entire scientific discipline, suddenly this approach is not working so well.

Image: Adi Holzer, via Wikimedia Commons

I had to learn this the hard way applying for funding when reviewers couldn't quite place me and
questioned my qualifications to perform techniques I've been using for years. Despite having worked on a particular approach for a long time, I wasn't as prolific in publishing about it since I was building a larger story, and reviewers didn't believe I could do it. So I put a portion of the story together and published it, but they still said it wasn't enough. I looked at what I had, at my submission deadlines, and decided to break things apart a bit more. It broke my heart a little as I was cannibalizing another paper in progress to break it into smaller "single approach" pieces. Now, while I write yet another paper, which relies on some of the previously published data, I so wish I had kept everything together! It would have been so beautiful and cohesive, and now I have to do somersaults to make my point. There is another paper which has been in the works for 10 years now (yup, ten) because we have been learning new techniques which are taking years to perfect, and I wonder if it will be worth it.

I watch people who stuck it out and built one of these beautiful and cohesive mechanistic stories and I have to admit I am a little envious. Not necessarily because of the high-impact paper and the admiration of their peers, but because of the pride that comes with having a great piece of work to call your own. In giving a talk I can still place all the smaller pieces into a bigger picture. Yet, I cannot tell if it is as evident for others such as study section members, search committees, and university administrators, to see. In the end, my belief has always been that one has to strike the right balance between publishing a large story and not taking too long to do it and risk appearing unproductive. I still have not figured out how to walk this tightrope...