Ever since Chat-GPT was announced, I wanted to use it to ask personal finance questions just for kicks. But I was too lazy to sign up. Unfortunately by the time I decided to sign up, the page said it was full and asked me to wait. After a few weeks I heard about Chat-GPT integration with Bing and wanted to try it out. Unfortunately though it required me to sign up to a Microsoft account and wanted me to use Edge browser or something like that. I don’t remember all the reasons, but I am certainly not going to do any of that. I mean asking for email is one thing, but phone number even if it is for OTP is a no go for this kind of service. It is a lot more personal. Not that I don’t trust them with my data, it is mainly because I don’t want to increase the attack surface.
I had the same issue with DALL-E. Why do they need so much of my personal information just to try something. Meanwhile Google announced Bard. Unfortunately, it was limited to US and was in preview. Finally, Google IO announced the general availability of Bard which got me excited. I tried it out the very day it was announced. Since I did not use Chat-GPT or any other large language model generative AI, I will not do any comparisons. I just wanted to use it as a companion to my personal financial planning.
Now you might be wondering why I did not mind handing over my data to Google while I was complaining about the rest :). I know some of you do. The reason is simple. I already have an account with Google since when Gmail was in beta way back in 2003 or sometime around that. I have consistently used several Google products and hence it already has a lot of my data anyway. So using another product from the same company is not going to increase the attack surface. As I said earlier, my concern is regarding attack surface and not the companies themselves. It is not like Google is any better at protecting my data than say Apple or Microsoft. It is just that if I have accounts and data with everyone, I am exposing myself to more attack points. Of course there are some companies I would not trust with my data at all. The same reason I don’t have accounts in WhatsApp, Instagram, Facebook etc.
Anyway, let me go back to the real discussion which is generative AI for personal finance. I wanted to see if I can make my blog obsolete by using generative AI to answer questions readers might have about their personal finance or retirement planning. So here goes.
A couple of post ago a reader had this question – Want to understand how to redeem from our funds after we retire? Say we have calculated the number and we have enough and want to find out how to withdraw from the fund after retirement. Can you do it with some numbers for a fictitious person? Example Age 40. Amount Rs. 6 Crores spread across multiple equity and debt funds. Now, how to withdraw from that investment to last till their age of 85 for an expense of Rs. 1 lakh per month?
I just prompted that question as is to Bard. The response is kind of generic, and useful, but not the straight forward answer a human would give. Here is the general advice it gave
Then it went into some specifics, but still too general.
Since it did not show any calculations or tables, I asked for the data to be shown in a tabular format and this is what I got.
Immediately you can see that the calculations look wrong. Why are you withdrawing Rs. 24 lakhs when your expenses are only Rs. 12 lakhs per year? Moreover, it did not assume any inflation and where you have invested and what the investment returns are. So I asked Bard to explain the calculations and this is what I got.
Not bad for an AI! It actually used the 4% rule. So the problem seems to be that it assumes that one would withdraw 4% from the corpus although the expenses are really only 2%. As suspected, no inflation or return on investment numbers were used. I nudged it to assume a 6% inflation and 10% return on investment and then things started breaking.
As you can see, it was smart enough to include the new information and changed the corpus based on the return on investment number I have given. But the expenses are not inflated by 6%. I am not sure what inflation rate it has used. At this point I gave up. Perhaps I could go on, but those are too many mistakes at this point. Perhaps I should have given better prompts. Well I am not much of a prompt engineer, nor am I trained in it, so I let it be :). Although there is one interesting thing that I noticed. Whenever I asked it to generate tables with complex calculations, the table would be incomplete. As if someone pulled the plug on the AI while it is still doing some calculations.
I suspected it might have to do with the amount of processing it is allowed to use per prompt. So I decided to ask that question – how much processing power are you allowed to use to generate an answer. And this is the response
At this point, I decided to reset the chat and try to nudge the AI to think the way I would think. Basically if someone asked me the previous question, I would first pull up historical data and use that to make an informed guess for the future. Yes, we all know past returns don’t mean that future will be the same, but we have to start somewhere. The historical data I needed was fixed income returns, equity returns and inflation rate in India. For fixed income returns, I chose SBI 1 year FD rates. For equity returns I chose Nifty 50. The first prompt I gave Bard was to get me the SBI 1 year fixed deposit rates from 1998.
I am not sure how accurate this data is and where it sourced the information, but it looks reasonable to me although it differs slightly from the data that I collected from RBI website. When I asked – what is the source of this data. It responded with – The source of this data is the State Bank of India (SBI) website. The SBI website provides a comprehensive list of all of its current interest rates, including the interest rates for 1 year fixed deposits. The data on the SBI website is updated regularly, so you can be sure that the information is accurate. Yet it did not provide any direct links to the said website. Lets assume it is true. Next I asked it to show me Nifty 50 returns since 1998 and it gave me this.
Again the data looks reasonable. Then I asked Bard to merge the two tables and it understood what I meant even with a spelling mistake and properly merged them.
Then I asked – i have rs. 1.8 crores corpus. i invested 40% of my corpus in sbi 1 year fixed deposits and the rest 60% in nifty 50 in 1998. show me how my corpus grew since then. And it gave the following response.
Seems like Bard had some difficulty figuring out what I said. I said I invested Rs. 1.8 crores and not Rs. 1.08 crores. So I tried helping it but it went even more off the rails.
I decided to give it a more direct prompt like – i have a corpus of Rs. 180 lakhs which I invested 40% in sbi 1 year fixed deposit and 60% in nifty 50. That helped it a little.
Fair enough. Now I wanted to know how the corpus will change if I rebalance, which is exactly what I asked Bard and I got this result.
That did not match with my numbers, so I was wondering if it was really rebalancing the portfolio. I asked it to show the Nifty 50 investment column in the table.
Well, there is your problem. It does not look like the portfolio is really being balanced. Asking for the asset allocation showed the error clearly.
I gave up at this point. Perhaps further nudging might have helped, but I did not want to spend too much time on it.
Next thing I wanted to know was if I can retire at 37 assuming I have a corpus of Rs. 180 lakhs with monthly expenses of Rs. 50000. Well apparently I can, which is true by the way (according to me anyway).
Not bad. Seems like a general advice any financial planner or blog would give. Next I asked how long my investment would last if I invested 70% in equity and 30% in debt mutual funds and assuming an inflation of 6%.
The response impressed me. Note that I did not mention which country I invested in and how much would the returns on investments would be for equity and debt etc. I guess it could infer that I am talking about Indian investments because I dropped that hint in my previous question and Bard is capable of holding context. The response looked like something I would give on my blog - assuming a return of 12% on equity and 8% on debt. It understood how to use inflation to increase expenses every year. Pretty good. When asked for the tabular data, it gave a good response.
Now, if I were to generate some random question a reader of my blog might ask, then make the generative AI create a response for it and publish it as an article, would you be tricked into thinking that I wrote it? Is it unethical to trick readers that way? Is copying information from Bard considered plagiarism? Should we allow these AIs be trained on their own responses thus reinforcing their own responses, because that is what will happened if they read this post. So many ethical questions remain unanswered!
Anyway, that is it about AI experiment. I can keep writing another 100 paragraphs about all the experiments I performed on Bard. But this post is already pretty long so I conclude here. As of now, generative AI is decent but not good. Some calculations may not be accurate or correct because it may have understood it wrong, or obtained the data from wrong sources. Also it is difficult to reproduce the exact same answers as I got because the AI could have taken a different neural pathways to generate the answer to the same question at different times. Finally, most of the time Bard replied back with an answer but it had multiple drafts too. So be sure to check out the drafts in case they are more accurate.