Chalfie grew up in Chicago, Illinois, son of the guitarist Eli Chalfie (1910—1996) and owner of an apparel store Vivian Chalfie (née Friedlen, 1913—2005). His maternal grandfather, Meyer L. Friedlen, immigrated to Chicago from Moscow at an early age; his paternal grandparents, Benjamin and Esther Chalfie, came to Cincinnati from Brest-Litovsk.
He matriculated at Harvard University in 1965, intending to be a math major, but he switched to biochemistry because it combined his interests in chemistry, math, and biology. He spent the summer after his junior year working in the laboratory of Klaus Weber at Harvard, but "It was so disheartening to completely fail that I decided I shouldn’t be in biology." As a result, in his senior year, he completed his major and took courses in law, theater, and Russian literature.