The ability to learn new concepts by recombining existing ones through symbolic instructions allows humans to reason about abstract concepts like the universe or mathematics. But this idea of “compositionality” remains a challenge in AI research. In our new paper, we propose a novel theoretical approach to address this problem. We also demonstrate a new neural network component called the Symbol-Concept Association Network (SCAN), that can, for the first time, learn a grounded visual concept hierarchy in a way that mimics human vision and word acquisition, enabling it to imagine novel concepts guided by language instructions.