The road to buying or selling a home is often a long one. Along the way, you may discover that your relationship with your real estate agent just isn’t working anymore. Maybe the agent is moving faster than you’d like. Or they’re not as available as you need them to be. Maybe they just don’t get you.

Now what? Is it OK to break up with your real estate agent? And if so, how can you gracefully end it? The answer depends on whether you’re working with an agent as a buyer or a seller.

Advice for Buyers

Real estate agents earn their commissions from sellers, which are split between seller’s and buyer’s agents. As a general rule, as a buyer, you won’t be asked to enter into a contractual or financial agreement with a real estate agent. Though there are some agents who ask buyers to sign an exclusivity agreement, in which the buyer agrees not to work with another real estate agent but this isn’t the norm.

Instead, a buyer makes a handshake agreement with the real estate agent. You’re basically agreeing to exclusively rely upon that agent. And that’s fair. Agents often work hard and spend a lot of time engaging with buyers, watching the market, writing contracts, showing properties, reviewing disclosures, and so on. Imagine how they’d feel after spending months working with a client, only to be informed that another agent found them the home they want?

If you’re not quite ready to be tied down to a particular agent, it’s better not to engage one until you are ready. That said, it’s OK to look for agents even if you’re a year away from making a serious move into the market. Some buyers need data and advice as they tip their toes into the market. Just be honest.  A good real estate agent will read your situation well and provide the appropriate amount of attention. They’ll act as a resource and be available when you need them. A big turn off for buyers is an agent who puts on the full court press when they are not nearly ready to buy.

Before you enter into a handshake agreement, do your homework. Ask friends for references or check out online agent reviews. Going to open houses is a good way to meet agents who work in the neighborhood you are looking to buy. Make sure you’ve picked the most compatible agent you can find. Start slow and feel out the relationship. It is harder to break up with your agent if you have too deeply engaged them.

As time goes on, you may find that the relationship just isn’t working. If that’s the case, just be honest and upfront before too much more time passes. Offer the agent constructive feedback about why it’s not working for you, and give them examples. It’s possible after an open dialogue the agent will do a better job for you. If not, at the minimum, you’ll have given the agent important feedback, which will help them do a better job for the next client (in theory, at least). This way you won’t be hitting the agent out of left field with the sudden announcement that you’re buying a house through another agent.

Advice for Sellers

Since the seller pays the real estate agent’s commission, the seller generally is required to sign a legal agreement with an agent’s brokerage firm. In most cases, the agreement is for six months. During that period, you’re contractually obligated to work exclusively with the agent and brokerage firm specifically on the sale of your home. In fact, even if you find a buyer on your own, (such as a friend), the listing agent/brokerage firm is still due their commission.

Just as a buyer must do his homework, it’s even more important for a seller to do his research, given the contractual agreement required. But you do have some leeway. Even if you sign a listing agreement, you aren’t necessarily obligated to sell your home. You can say no to open houses or showings and you aren’t required to sign a contract with any buyer. This is where a seller still holds some power.

However, most listing agreements state that if the listing agent brings an offer at the listing price and the seller doesn’t accept it, the agent is still due a commission. This scenario happens sometimes when the listing agent and seller aren’t getting along.

In most situations, if the listing agent isn’t doing a good job but there’s still time left on the agreement, you should simply tell the agent it’s not working out. A good, fair and honest agent will apologize for not meeting your expectations and will agree to release you from the agreement ahead of schedule. The agent doesn’t have to do this, but it would be ridiculous — not to mention counter productive — for an agent to insist on enforcing the agreement when the client isn’t happy. It does happen, however. Usually the seller responds by no longer agreeing to open houses or considering offers from the agent.

Sometimes, an agent wants to break up with the seller. Maybe the seller insists on keeping the price of the home too high or isn’t cooperating to accommodate showings. The agent simply feels she can’t be successful with the seller, no matter how much time she puts into the job. Here again, the two parties are bound by a legal agreement, which can be terminated earlier by mutual consent. If you’re a seller whose agent wants out of the agreement because you aren’t taking the necessary steps to sell your home, it’s best to let them go — and to give serious consideration as to whether you’re really ready to sell or not.

Brendon DeSimone is a Realtor® and real estate expert based in San Francisco and New York. He is a contributor to Zillow Blog, has collaborated on multiple real estate books and is often quoted by major media outlets. Follow Brendon on Twitter.