At what point is it statistically better to take Tavern Brawler over an increase to Str or Dex if 1/3 of all your attacks are made with an improvised weapon?

Example: A Thief rogue 4/Champion fighter 5, with 16 Dex.

You make two attacks a turn with the Attack action, and one (throwing an acid vial) with the Fast Hands thief feature:

Two attacks with dagger: +7 to hit 1d4+3 (6)
If one of daggers hit assume sneak attack applies 2d6 (8)
One attack with an acid vial: +3 to hit 2d6+3 (11)

Statistically, would it be better to take Tavern Brawler for all 3 attacks to be at +7 to hit?

Or should I put the ASI into Dex for two +8 to hit and +4 to hit?

The two options for you are actually stunningly close, statistically.

Mechanically, Tavern Brawler is statistically better if you are fighting enemies with an AC of 18 or above, while the ASI is better for AC 17 or below. However, the difference never amounts to more than 0.4 DPR.

Link to statistical analysis: https://anydice.com/program/1343d

My analysis assumes that you are always attacking without advantage, but that sneak attack is always possible. If you have advantage on your first attack, you will likely be better off with the ASI. I also assume that you are not attacking with STR or that the +1 to STR/CON from Tavern Brawler would not round out your STR score to an even value.

Because these two options are so close in combat, it may be worth considering out of combat mechanics as well. Vials of acid are both costly and consumable, so they may not always be available. Additionally, putting an ASI into DEX would improve your AC, stealth, sleight of hand, and many other useful abilities.

On the other hand, Tavern Brawler does create many useful roleplaying opportunities.

tl;dr The two options are nearly identical in combat, so other abilities and character traits should be the basis of your decision.

The proficiency bonus is the most important factor.

The following is a comparison of average damage per round assuming 2 attacks are made with a dagger and one with an acid vial, while sneak attack damage is not taken into account. Both the ASI and the Tavern Brawler feat provide additional benefits that don't factor in this damage comparison.

The ASI is somewhat better when your proficiency bonus is 2, 3, or 4; whereas Tavern Brawler and the ASI are roughly equivalent when the proficiency bonus is 5 or 6.

To my surprise, the damage dice of your primary weapon is insignificant for this comparison.

For example, the rogue 4 / fighter 5 (proficiency bonus is 4) would be better off choosing the ASI when it levels up to rogue 4 / fighter 6.

To arrive at this conclusion I created this graph which plots the average damage per turn against the target's armor class.

If we were to consider a a.c. 17 monster
Taking tavern brawler for +7 on all three attacks would thus make it so you hit on a 10 or higher on d20
Meaning 11/20 sides on d20 would be a hit or 55% to hit for each attack
The damage would be multiplied by .55
To all three attacks at total 17.5

Taking asi increase to dex makes the hit
A 9 or higher on d20 for daggers 12/20
And 1d4+4 damage each. 13.2
Vial to hit +4 or 13 or higher on d20 8/20
2d6+4 damage (12) 4.8
Total 18

Increasing asi increases accuracy on dagger attacks and thus applying sneak when applicable
Also increasing asi increases damage of all 3.

So in final if more than half of attacks are made with improvised weapons take tavern brawler. If more are made with normal weapon attacks take the asi.

Not sure if my math is correct but this is where I got equation How do you calculate average damage for DPR comparisons?