You can use tasker (if you have spare android device). My lights are on china server and voice commands work through IFTTT and tasker. I decided to bypass the Google Assitant integration, and use IFTTT, AutoNotification Tasker plugin to do it. The main objective being bypassing Xiaomi's servers. The way it works is that Google Assistant is set on a key word. I chose to use "can you". So when I say, "OK, Google, can you..." it knows to send this text i said to the IFTTT. Then IFTTT has a small applet which sends notification to my tablet, with the exact text I said (excluding "OK Google"). AutoRemote then intercepts this notificaton, passes it to Tasker, which parses the text and does what needs to be done.
If sounds complex, but it is really fairly easy. All I had to do is check for certain words, i.e. when it finds the words "left lamp" in text, it knows that I want to do something with left lamp. If it finds the word "percent", it looks for previous word, and knows that this is a brightness level for the bulb. If it finds words "red, blue, green" etc, it knows that is should set the bulb(s) to this color. Then there are words "on, off, switch, toggle", etc, etc. And it works quite well.