Daniel Hawkins built this house in 1810 and had three daughters, but women could not inherit property and he left the house to a grandson, James Brown. The three-bay house had an extension off the back and a front porch added later in the 19th century. The rear extension and front porch have been removed. During the late 19th Century and early 20th Century the house was rented. Belle J.E. Rowell rented the house in the late 19th and early 20th Centuries. A widow with several children, she became a photographer and converted images of local sites into postcards which she sold to help support her family. These postcards have added to the knowledge of how Miller Place looked in the early 20th Century. In 1939 William McCaddin of Brooklyn purchased the house and used it as a summer retreat. Rose (McCaddin) Lynch was the last person to live in the house.
In 1998 Arthur and Linda Calace purchased and donated the house to the Miller Place-Mt. Sinai Historical Society. It is currently used as host to the Annual Harvest Arts Festival and will be used for local artists and for displaying historic picture galleries, etc.. as well as undergoing extensive repairs and renovation.