Inclination, or Memory?

When does something to which we are drawn represent not merely a "liking" for "x," but rather an infant's lost memory of that which was once deeply familiar?