If a listing is dominated by Amazon and has three or four other FBA sellers, is it worth going for that product or not?

Additionally, if we offer a lower price than Amazon, can we win the Buy Box, even if only temporarily?

There’s not enough information to say yes or no. Never expect a correct answer when only a few details are provided. Analyzing a listing is a serious task, and you need to review all the data to determine if it’s a good purchase or not.

In short: Can you compete with the best offer? What does the price history indicate? And so on.

First, confirm if those sellers are authorized sellers or if there were other sellers in the past. Gather some information about the brand as well. I also agree with @masud530 that without proper details and graphs, we can only provide suggestions based on our prior experience.