Construction on Fourth Street and its impact on the site’s parklet, the business’s owner said, have driven away customers in the critical early months of operation when he needs to establish his presence in the neighborhood.