See, it's a fiction film and top of that a sci-fi superhero film. So, you should surely stop looking for logic. For story, you can get it if you have watched all the major film of MCU. Anyways, you have all the rights to dislike the story if it doesn't click with you and your criticism should be based on disliking of that story or its execution. Like all the Time Travelling stuff didn't click with me, it was really detachable for me, but its okay, I was more expecting a film that'd define the term 'Summer Blockbuster' and Endgame is that film. Entertainment is factor and that sometimes push everything asides.