The Wood Tent: “The thing that appeals to me” about the houses of architect Horace Gifford, Charles Renfro says, “the thing I recognized as his calling card, is parallel operative glass sliding doors so that a room can always be open almost in its entirety to the outside world.” Photo: Annie Schlechter

Charles Renfro had been renting a Cape Cod–style bungalow with friends in Cherry Grove on Fire-Island for a few years when he decided to look at buying a place in the Pines, the town next door, which is “decidedly more design-y, and more sexy, and more intimidating,” he says. “Part of why it was intimidating is because of the houses,” many of which are modern and open and put life on display like a stage.

“The master of that was Horace Gifford,” Renfro adds, the handsome Louis Kahn acolyte who, from 1961 until his death in 1992, built more than 60 houses on the island. Gifford’s houses started out simple and sustainable, although no one was calling them that back then, and respected the scrubby landscape. In the summer of 2013, this “postage stamp of a house” by Gifford came on the market. “The owner had died. Sadly, he had occupied the house until his death. He was so in love with that place.” Renfro bought it. Since then, “I think we have kept the previous owner’s spirit around. I think he’s enjoying being with us in the house even though it’s a very different place than when I got it.”

Renfro is a partner at the architecture firm Diller Scofidio + Renfro, known for the High Line, the Shed, the Broad museum in L.A., and the rethink, a decade ago, of Lincoln Center. His studio recently built Blue Dream, an 11,000-square-foot organic swirl in East Hampton that calls to mind Eero Saarinen’s TWA terminal at JFK and John Lautner’s dramatic Arango House in Mexico. Renfro’s Pines place reflects a simpler side of his aesthetic. “My house is as minimalist as you could possibly make it,” he says. “There is not a spare nail in the wood. I mean, in a way, it just goes away.”

Gifford was just 28 when he built this one. It was “one of three he made that were identical, and they were all made at the same time,” Renfro says. One has since burned down, and the other has had rooms added on. But when he bought it, Renfro’s was basically unchanged. “There was nothing in the backyard. There were no decks — I mean there was a little perch that was rotting,” he explains. “But the bones of it, what I call the ‘wood tent,’ they were good.”

Renfro’s own work on the house has been deferential. “I didn’t change the envelope,” he says, “although I re-sided the entire thing.” He even kept the original sliding doors: “They are a little bit rusty, but they are operational.” In collaboration with architect Anne Nixon, he put in a new kitchen, redid the bathroom, and added the deck out back. The bedrooms were tiny and remain so. “Horace believed houses were for living together communally, indoors and outdoors, as much as you can,” Renfro says, “and bedrooms were for, pardon my French, fucking and sleeping, and that was it; you didn’t lounge around.”

One of the reasons he bought the house was that it was on an underutilized lot. But instead of adding a wing, Renfro “made everything outside,” he says, with a “runway of deck that is the exact same width as the living room and that extended out into nature and then the nature itself was lush and overgrown and all local species.” In 2015, he added a pool, leaving one edge cut off, a beveled waterfall.

There is no air-conditioning, and it is not winterized. Renfro shares the house with his partner, concert pianist Daniel Gortler, as well as their friend Alexander Galan, who likens it to a birdhouse: “We’re like migratory birds. We come here on the weekend and then we leave.”

Renfro hung text-based artwork in the house because he finds the Pines “just a visually overstimulating place, and it’s intellectually an under-stimulating place,” he says. “And always the pieces that I have chosen are provocative; they try to stitch together those two things. Then I thought it would be a kind of wonderful mise-en-scène for all the activities.”

The house is rarely empty; friends and friends of friends are always coming and going in the day and at night, and impromptu parties start up at all hours, sometimes with DJs, always with some degree of abandon. But it’s also tucked away, invisible to passersby on the boardwalk who just hear music and the laughter.

The Outdoor Rooms: “I did not take a thing down” when adding the deck, Renfro says. “When it hit a tree, it surrounded the tree.” The glass dining table replaced a similar one that had come with the house but mysteriously disappeared off season. The see-through coffee table was also inherited. The quick-dry outdoor couches are from Crate & Barrel. Photo: Annie Schlechter

The Living Room: Pictured, from left, are works by Andrew Brischler, Luke O’Halloran, John Giorno, David Adamo, Scott Reeder (no, it’s not actually a chalkboard), and Maynard Monrow. All the art is text-based, which Renfro “thought would be really interesting on Fire Island because our eyes are so full of imagery, of men, of nature,” he says. The sectional is custom by David Burke, the chair in the middle is by Poul Kjaerholm from 1952, and the one on the left was made out of bicycle-frame parts by the artist Max Lipsey. The rug is from Article. Photo: Annie Schlechter

Renfro’s Bedroom: The house has three bedrooms, each not much bigger than the size of a bed. The small side tables are by Kartell, and the yellow USM wardrobe visible in the mirror functions as Renfro’s work desk when he’s out there. The painting with the stars is by Bazooka Joe; the print on the right is by Piotr Szyhalski. Photo: Annie Schlechter

Brian Roettinger’s 2013 prints, You Complete Me, hang in Alexander Galen’s bedroom. Photo: Annie Schlechter

The Kitchen: The house was renovated with architect Anne Nixon. The custom island and cabinets are made of Douglas fir, the upper doors painted in a pattern that Nixon created. The felt-covered steel bar stools are from ABC Home. Renfro designed the stainless-steel bar light fixture and had it fabricated by Lites on West. Photo: Annie Schlechter

The Bathroom: Renfro, with his contractor Don Kolhepp, built a powder room in a shed next to the deck. The wall covering is a large-format printed ceramic tile by Paper 41. The sink is by Alape. Photo: Annie Schlechter