I did a Google search to find out factory drawbacks, but did not succeed. Please invest a few minutes to clarify what you meant by 'not good'.
I'm willing to accept your decision without further questions if it is well reasoned. However, I can't see any purpose why we should keep the existing instantiation process without considering alternatives.
If a factory method not only creates a new class but also configures it from an argument, you then have the ability to customize the configuration process. It also means that external classes know less about the components' internals, which is always good, even if subclassing a component takes a bit longer.