The run-off will be held in two weeks.
Data from the Central Electoral Commission, received from all 40 polling stations, show that Haase, a 58 year-old lawyer, serving on Šakiai District Council, received 33.8 percent of votes, and Bastys, a 53 year-old former MP, got 19.4 percent.
Agriculture Minister Giedrius Surplys, representing the ruling Lithuanian Farmers and Greens Union, came in third with 16.6 percent, and Paulius Visockas, representing the opposition social democrats, got 12.9 percent of votes.
The turnout stood at 29.8 percent.
The by-election in Zanavykai constituency was held after its representative in the parliament, Bastys, renounced his mandate following a failed impeachment and a large protest outside the parliament.