I’ve said it before, and I’ll say it again: Using unisex names in written fiction is a bad idea. It opens up a door for reader confusion, and the writer should always aim to avoid reader confusion. (This doesn’t mean you don’t keep details from the reader when it serves the plot, it means what you show is clear.)
Still, I can’t go as far as to say that you shouldn’t use any name with a history of being used for both genders, as that would exclude an enormous amount of names from use.
As you probably know, Ashley and Leslie and many other names were once boy names that are now mostly girl names (in the US; Ashley is still mostly boy in the UK). But if you look at the baby name lists on the Social Security website you will learn that many other names that are considered girl names were once used for boys as well, names like Elizabeth and Marie.
I would never say you shouldn’t name any characters Elizabeth or Marie because readers might be confused as to what gender the character is, unless you plan to name a male character with those names. In which case, don’t.
The important thing with naming is the perception of the majority of the potential readers. If almost all readers alive today would consider the name a female’s name, then you should use it that way with impunity.
If, however, the name you want to use is Lou, a name that has been most often used as a girl’s name and yet more often perceived as a boy’s name, you would be best advised to choose another name—or, at the very least, to use Lou interchangeably with a “long name” that is gender specific (Louis or Lucy, perhaps) or a gender clarifying nickname (Buddy or Princess, or anything that points clearly to one gender).
Writers of contemporary realistic fiction should use their best judgment to choose names that will be clear. Writers of Fantasy will have to work a bit harder at this.
For Fantasy novels, writers often chose more rarely used or archaic names. If these names have a history of being used as boys or girls names either in one culture or in different cultures, the writer will have to judge how the name will be perceived to contemporary readers.
Names ending in O will generally be considered male, while names ending in A will generally be considered female. There are other sounds that are more often male or female in names that should be considered when deciding whether to use a name for a male or female character, but there is an easier way to do this.
Think about the name you want, using for example the name Edme from this week’s naming. This name, as I mentioned, is very close to the rising in popularity Esme—a clearly female name. I think the fact that Edme is so close to a current girl name would make the majority of readers think the character is a girl.
So what name are you considering for your Fantasy character? Think of what names which are on the popularity charts now that are most similar to that name, and what gender those names are most strongly associated with. Now you have your answer.