The retailer fled Nazi Germany as a child and bought an existing men's specialty store in Greenwich, Conn., in 1956.