Congregation B'nai Israel had its beginning with the Rockville Hebrew Ladies Association. This dedicated group of women originally held services in the home of one of the members. In 1931, they bought the Greek Revival building on East Main Street for $1,200. They were able to turn the dilapidated building into a synagogue, community center and school. Then, in 1946, the congregation purchased "Castle Sunset", the building we're in now.

Castle Sunset was built by Samuel Brown in 1894. It was originally owned by Alvah N. Belding,  a textile manufacturer. He was the vice president of Belding Brothers, an internationally known thread manufacturer. Belding Brothers owned textile mills in Canada, California and Belding, Michigan, as well as in Rockville, This building was Belding's private home.

Castle Sunset is a typical Queen Anne style house. The variety of materials, irregular roof line and diamond glazing of the second floor windows are characteristic of this style. The third floor, originally used as a ballroom, is still in its original state, with huge chandeliers and intricate paintings on the ceiling. Across from the synagogue is the carriage house, which is now used as a recreation hall. The grounds, originally consisting of five acres of land located at the top of a slope, had an excellent view of the Connecticut River Valley. The property is now surrounded by a simple iron fence and has been planted with specimen trees.