She is perhaps saying that she doesn’t want to remember something or could she be saying that she doesn’t want to remember something alone, that she wants to share her memories with the other people of the “House”. A life lived with no people in it is not a life well lived, thus memories should be populated with the memories of the people we care about.
This poem is similar to “The Angle of a Landscape” in that she is talking about memory as being like a “House” – in that poem she talks about the walls of her room and here it’s the physical architecture of building a house. Yet the “House” in this poem feels empty because there is only a “Mouse” in the “Garret” that is either eating the trash left behind, or is making its own “Refuse” in the form of its droppings. But she is also referring to the grave when she describes “the deepest Cellar / That ever Mason laid” which gives this “House” a foundation of the all the dead who ever once lived there and perhaps this “Mouse” is gnawing on the dead themselves.
The etymology of the word “Cellar” is worth noting in that the word has sometimes been confused with ‘sollar’, the upper room of a house (the solar), and Emily seems to be playing with this confusion by implying that when one is in the “Cellar” they are ready to be received into heaven, but she fears that “Almighty God” might forget about her and leave her alone in the grave. Her anxiety here is that she does not want to be forgotten when she dies and she doesn’t want the house she built (perhaps her life and life’s work) to be forgotten.