I recently deployed my .NET 6 application to Microsoft Azure. The app uses NHibernate for working with SQL Server database. After some time, I started getting an unhandled exception when opening the application:
NHibernate.Exceptions.GenericADOException: could not load an entity:
...
---> Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myappdb' on server 'myapp.database.windows.net' is not currently available. Please retry the connection later. If the problem persists, contact customer support, and provide them the session tracing ID of '{EB501CF2-2F21-4E28-9042-2B83EEE57B91}'.The database is not currently available was very interesting to me. I struggled a bit with solving that, so if you want to know how I did it – continue reading 🙂
The problem
My .NET 6 app is an Azure App Service, and it uses SQL Server database. Quite standard Azure stuff.
The issue was that when opening the ASP.NET web app after some time of inactivity, there was an HTTP 500 error. When I checked the logs, the full error looked as follows:
NHibernate.Exceptions.GenericADOException: could not load an entity: [MyApp.Infrastructure.DomainObjects.ApplicationUser#37931c5a376748d7a49f215ca30283a2][SQL: SELECT applicatio0_.applicationuser_key as id1_3_0_, applicatio0_1_.UserName as username2_3_0_, applicatio0_1_.NormalizedUserName as normalizedusername3_3_0_, applicatio0_1_.Email as email4_3_0_, applicatio0_1_.NormalizedEmail as normalizedemail5_3_0_, applicatio0_1_.EmailConfirmed as emailconfirmed6_3_0_, applicatio0_1_.PhoneNumber as phonenumber7_3_0_, applicatio0_1_.PhoneNumberConfirmed as phonenumberconfirmed8_3_0_, applicatio0_1_.LockoutEnabled as lockoutenabled9_3_0_, applicatio0_1_.LockoutEnd as lockoutend10_3_0_, applicatio0_1_.AccessFailedCount as accessfailedcount11_3_0_, applicatio0_1_.ConcurrencyStamp as concurrencystamp12_3_0_, applicatio0_1_.PasswordHash as passwordhash13_3_0_, applicatio0_1_.TwoFactorEnabled as twofactorenabled14_3_0_, applicatio0_1_.SecurityStamp as securitystamp15_3_0_ FROM ApplicationUsers applicatio0_ inner join dbo.AspNetUsers applicatio0_1_ on applicatio0_.applicationuser_key=applicatio0_1_.Id WHERE applicatio0_.applicationuser_key=?]
---> Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myappdb' on server 'myapp.database.windows.net' is not currently available. Please retry the connection later. If the problem persists, contact customer support, and provide them the session tracing ID of '{EB501CF2-2F21-4E28-2342-2B836E35E891}'.
at Microsoft.Data.ProviderBase.DbConnectionPool.CheckPoolBlockingPeriod(Exception e)There were two characteristics of this error:
- it always appeared after some time of not using the web app
- I never reproduced it locally
So it must have been something wrong with Azure.
Transient issues in Azure databases
First thing I did, obviously, was checking Azure App Service logs. This is what I found:
The first recommended step got me curious. To reduce the impact of connection issues caused by future reconfiguration, please implement retry logic in your code.
So I started reading. There’s even official Microsoft documentation for that. It basically says that in the cloud environment you’ll find that failed and dropped database connections happen periodically.
I must admit it seemed quite new to me. However, I started digging into that. I used Polly to implement the retry logic in my app. I found few places where there was an NHibernate query for fetching users executed and wrapped it into the retry policy.
Nothing worked. I was still getting the error. Then, after some talks with my coworkers, we found the real reason behind that…
Solution: Azure database tier
When talking about the issue with my teammates, someone mentioned that the databases in Azure are serverless by default.
I logged into the Azure Portal and checked the “Compute + storage” settings my SQL database my app used:
Indeed, the tier selected was “General purpose (Scalable compute and storage options)” with “Serverless” compute tier selected.
I changed it to “Basic (For less demanding workloads)”:
and… it worked 🤯 All errors saying that the database is not currently available disappeared. I also removed all Polly-related code for retry policies.
A little explanation
Why do serverless databases in Azure behave like that?
I finally found this explanation in the Microsoft documentation:
If a serverless database is paused, then the first login will resume the database and return an error stating that the database is unavailable with error code 40613. Once the database is resumed, the login must be retried to establish connectivity
That blew my mind! It means that the database is not currently available exception is expected. There’s also another part there:
Database clients with connection retry logic should not need to be modified. For connection retry logic options that are built-in to the SqlClient driver, see configurable retry logic in SqlClient.
Apparently, users of ORMs like Entity Framework should not have to worry about those issues. Unfortunately, NHibernate doesn’t have such built-in transient errors retry logic. That’s why I experienced this particular issue and couldn’t find any straightforward solution online. Simply not many people (if anyone at all? 😁) experienced this before.
If you want even more details and context, check out this article: SHOULD I USE SERVERLESS FOR ALL MY AZURE SQL DATABASES?
What if I want to use serverless Azure database?
Well… if you really want to use the serverless features of Azure, you might be struggling with NHibernate. As I mentioned, apparently there’s no built-in transient failures retry mechanism in the ORM. There’s one promising solution here. However, currently it’s build for NHibernate 3.3.1.4000, so it doesn’t work with the current version. There’s a Pull Request and an issue opened for upgrading it to the newest version and .NET 6/7. Let’s hope it is done soon.
Otherwise, you can try using Polly as I did initially. However, I didn’t manage to make fully working.
If you have any other idea or solution – let us know in the comments below.
Conclusion
I hope this article saves you some hours of debugging, so you can quickly solve this database is not currently available error in Azure. I wish I found this article on another website few weeks ago 🙂
