Refresh Power Query With VBA

When I’ve finished building a solution in Excel, I like to give it a little polish, and really make it easy for my users to update it.  The last thing I want to do is to send them right clicking and refreshing everywhere, so I program a button to refresh Power Query with VBA.

The interesting part about the above statement is that Power Query doesn’t have any VBA object model, so what kind of black magic trick do we need to leverage to pull that off?  As it turns out, it’s very simple… almost too simple in fact.

A Simple Query

Let’s just grab the sample data file from my post on pulling Excel named ranges into Power Query.  Once we’ve done that:

  • Click in the blue table
  • Go to Power Query –> From Table
  • Let’s sort Animal ascending (just so we know something happened)
  • Next save and Exit the query

At this point, we should get a new “Sheet2” worksheet, with our table on it:

SNAGHTML34cf68

 The Required VBA Code

Next, we need to build our VBA for refreshing the table.  Rather than record and tweak a macro, I’m just going to give you the code that will update all Query Tables in the entire workbook in one shot.  But to use it, you need to know the secret handshake:

  • Press Alt + F11

This will open the Visual Basic editor for you.  If you don’t see a folder tree at the left, then press CTRL+R to make it show up.

  • Find your project in the list (It should be called “"VBA Project (Selecting Data.xlsx)”
  • Right click that name and choose “Insert Module”
  • In the window that pops up, paste in the following code:

Public Sub UpdatePowerQueries()
' Macro to update my Power Query script(s)

Dim lTest As Long, cn As WorkbookConnection
On Error Resume Next
For Each cn In ThisWorkbook.Connections
lTest = InStr(1, cn.OLEDBConnection.Connection, "Provider=Microsoft.Mashup.OleDb.1", vbTextCompare)
If Err.Number <> 0 Then
Err.Clear
Exit For
End If
If lTest > 0 Then cn.Refresh
Next cn

End Sub

NOTE:  The code above was updated 2015-08-19 to reflect the method posted here:  https://excelguru.ca/update-refresh-power-queries-with-vba/

Now, I’ll admit that I find this a little looser than I generally like.  By default, all Power Query scripts create a new connection with the name “Power Query –“ the name of your query.  I’d prefer to check the type of query, but this will work.

Speaking of working, let’s prove it…  But first, close the Visual Basic Editor.

Proving The Refresh Works

The easiest way to do this is to go back to the table on Sheet 1 and add a new row to the table.  I’m going to do that first, then I’m going to:

  • Press Alt + F8
  • Choose “UpdatePowerQueries”
  • Click Run
  • Go back to Sheet2 to verify it’s updated

If all goes well,  you should now have another row of data in your table, as I do:

image

Adding Polish

Let’s face it, that’s probably harder than going to Data –> Refresh All.  The goal here was to make it easier for my users.  So let’s do that now.

  • Return to Sheet 1
  • Go to the Developer Tab (if you don’t see it, right click the ribbon, choose “Customize Ribbon” and check the box next to the Developer tab to expose it)
  • Click Insert and select the button in the top left

image

  • Left click and drag a button onto your worksheet

When you let go, you’ll be prompted to assign a macro.

  • Choose “UpdatePowerQueries” and click OK
  • While the dots are still on the corners, click in the text
  • Backspace it out and replace it with something helpful like “Update Queries” (if you click elsewhere, you’ll need to right click the button to get the selection handles back.)
  • Click in the worksheet to de-select the button

SNAGHTML57873a

That’s it.  Test it again by adding some more data to the table then clicking the button.

Ramifications

I write a lot of VBA to make my users lives easier, and generally use this kind of technique as part of a bigger goal.  But regardless, it can still be useful as a stand alone routine if you want to avoid having to train users on how to do things through the ribbon.

Share:

Facebook
Twitter
LinkedIn

102 thoughts on “Refresh Power Query With VBA”

  1. This is really useful, thanks!

    If i try and refresh my PQ tables with VBA I often (but frustratingly, not always!) get the error that my database is opened by 'Admin' on my machine even when the database is closed. I can either try the macro again (sometimes it works) or I can refresh each table manually by right clicking on each one.

    Do you have any idea why this could be happening and what I could do to try and solve it?

    Many thanks,

    Alice

  2. Hi Alice,

    It depends on what kind of database it is. If it's Access, it could be that another user is in the file and has it locked. Can you share what database you're trying to connect to?

  3. Great code Ken

    I have several power queries that I will execute usingthis approach. However, I need a mechanism whereby I can lock out the user from interrupting the queries as they are running. When each one refreshes a message in the status bar indicates so but also 'invites' the user to 'ESC to cancel'
    Normally I would use Application.EnableCancelKey = xlDisabled to prevent users breaking into code but this doesn't seem to work here
    I want to able to indicate to user that queries are refreshing but also stop them from cancelling as it sends them straight into this underlying code

  4. Hi Anthony,

    The issue here is that when you kick off the connection refresh, Excel immediately moves on to the next refresh, and fires it as soon as the previous one is done. With no "AfterConnectionUpdate" event, there's really no way to set up a trigger to monitor the last refresh, short of using a Wait procedure. That's obviously not ideal, as we don't know how long it will take a connection to refresh.

    We could use code similar to the following to disable the cancel key until immediately after the last connection is called:
    Dim cn As WorkbookConnection

    On Error GoTo ErrorExit
    Application.OnKey "{Esc}", ""
    For Each cn In ThisWorkbook.Connections
    If Left(cn, 13) = "Power Query -" Then cn.Refresh
    MsgBox "Refreshed!"
    Next cn

    ErrorExit:
    Application.OnKey "{Esc}"

    I'll leave it up to you to decide if you want to adjust it to disable the Cancel key until x seconds after the last refresh is kicked off.

  5. Hi Anthony, try this

    Data->Connections->
    Select the connection->Properties
    Uncheck the box "Enable background refresh"

    Regards

  7. Hi Ken
    I've built quite a large data model which loads up around 40 data files, each transformed using PowerQuery then loaded into data model. If I had to manipulate all of these files by VBA I'd be at it for months. PQ has made a mountainous task possible
    The thing that concerns me is that should any of the data files change in anyway then the respective workbook query will fail. This will be seen by looking in the PQ window. Some data files come from external clients so there is every chance one day some bright spark will decide to add a column. As the PQ is tightly bound to the data source it will just fail
    I have a range of data files and I would like to loop through triggering a refresh on each of them, one by one. I'd like to be able to post a message with success/failure to alert the user that a query has failed due a problem with one of the data files
    There doesn't appear to be a way of triggering an indivual query refresh then have VBA wait for it to complete before proceeding to the next one
    I can trigger a mass refresh but then you would have to look in the PQ window to check that all of have successfully executed
    Cheers
    Anthony

  8. Apologies, you've kind of indicated already that I'm trying to achieve isn't possible!
    It would be handy if a query could simply post the status you see in the window to a place more visible to user. I.e capture the query success status in VBA somehow

  9. Anthony, if you're going to pull directly into the model with Power Query, and you're at all concerned about the stability of the data, I HIGHLY recommend you take a look at Chris Webb's blog on dealing with data source errors: https://cwebbbi.wordpress.com/2014/09/18/handling-data-source-errors-in-power-query/

    I'd be implementing this right away. Next thing you could do is create a simple measure to COUNTROWS on each table in the Data model, and return them to a cube formula or pivot. This would give you a good indicator if something went wrong. (1 record = bad, more than one = good).

  10. Hi Ken
    Keen to have the core workbook / thin clients arrangement but no PowerPivot server side
    I haven't checked but I'm wondering whether it is possible to use power query in a different workbook to connect directly into a data model in another?
    If not (and it would be a great additional feature) then my only option is to export some reporting data sets in csv form from my core workbook to be consumed by thin client workbook
    Cheers
    Anthony

  11. Hi Anthony,

    Unfortunately I don't think that you can connect to the data model in another workbook, no. But you could connect to the same data sources using other workbooks, or even connect to a worksheet from another workbook (allowing you to consume a PivotTable output from a PowerPivot model in that workbook...)

  12. Pingback: Power Query – Controlling M Query Function with User Driven Parameters | Devin Knight

  13. Hi Ken,

    Just wondering if you have extended this into the PowerPivot data model in Excel 2010. Meaning can you update the Data Model in PowerPivot via VBA where the connections are using PowerQuery rather than direct connects to the sq

  15. I used Power Query to pull data from another excel file, transform it and then use the data to build a Dashboard.

    I need to send this dashboard out to 10 leads of 10 different business units with each having just their data every week. Their file should NOT have any other data present at all in the file.

    I'd like to be able to create the 10 different versions and maybe even send the mail out to them on 1 click. I can probably do that by filtering out the data in the table through VBA and refresh the pivots I have in the dashboard.

    Is there a way though using VBA, to tell power query to refresh data for a particular business unit?

  16. Not purely through VBA no, but if you pull in a parameter table, then you can read from Excel cells to dynamically filter the query. That would give you the ability to - via VBA - update the parameter cell, then refresh the query, giving you just that department.

  17. Super helpful thanks!!
    it was useful to help me find issues with my Long Running PowerQueries

    Public Sub UpdatePowerQueries()
    ' Macro to update my Power Query script(s)
    Dim TStart As Date
    Dim TEnd As Date

    Dim cn As WorkbookConnection

    For Each cn In ThisWorkbook.Connections
    If Left(cn, 13) = "Power Query -" Then

    Debug.Print cn
    TStart = Now
    cn.Refresh
    TEnd = Now
    Debug.Print CStr(DateDiff("s", TStart, TEnd)) + " Seconds"
    Debug.Print ""
    Debug.Print "----"
    End If
    Next cn
    End Sub

  18. Did not work for me. I have a Table called TABLE1 on sheet1 in a workbook. In the same workbook on sheet2 I have a table connected to a power query for TABLE1. I went to sheet1 and added a row and then went back to sheet two and ran this code. Nothing happened. the connection refreshed but the table based off the query did not update. I'm running Excel 2013 with Power Query with all the latest updates. Sigh.

  20. "Not purely through VBA no, but if you pull in a parameter table, then you can read from Excel cells to dynamically filter the query. That would give you the ability to - via VBA - update the parameter cell, then refresh the query, giving you just that department." - Didn't quite understand this, especially the "dynamically filter the query" part. Can you elaborate or point me to any blog post that explains the technique?

  22. Can this code be modified to run in a protected worksheet? Have tried running an unprotect code, then this code, then another to re-protect the worksheet but it never works for me. Even when adding delays in the code.

  23. Hey Carl,

    My thoughts are that the query is not finished loading before the protection is re-applied. What I'd try is re-protecting the workbook in VBA, but adding this to the end of the line:
    , userinterfaceonly=true

    That should set the protection to protect against users doing something, but still let the macro run.

  24. I am trying to achieve auto-scheduled update, by using Task Scheduler and another Workbook with macros.
    And the problem is - macros alone in the workbook with connections works fine, when I try to run this macros from another workbook by using Application Run and then check works it says "Download did not complete" for each of the query, despite computer was loaded in full, as when I update all manually or with macros in right workbook. Do you have any workaround?

    Macros in Model - workbook with queries:
    Option Explicit

    Public Sub UpdatePowerQueries()
    ' Macro to update my Power Query script(s)

    Dim lTest As Long, cn As WorkbookConnection
    On Error Resume Next
    For Each cn In ThisWorkbook.Connections
    lTest = InStr(1, cn.OLEDBConnection.Connection, "Provider=Microsoft.Mashup.OleDb.1")
    If Err.Number 0 Then
    Err.Clear
    Exit For
    End If
    If lTest > 0 Then cn.Refresh
    Next cn

    Range("J7").value = "Updated - " & Date

    End Sub

    Macros in workbook "Macros" - that will be scheduled:
    Option Explicit
    Sub Auto_Open()
    '#### To stop the macro you can use keyboard = "ESC" ####'
    Call Main
    End Sub

    Sub Main()
    'Disabling some options for better performance during the macro execution'
    Application.Calculation = xlCalculationManual
    Application.ScreenUpdating = False
    Application.DisplayAlerts = False

    Dim strRootFolder As String
    Dim strSearchPath As String
    Dim strSearchName As String

    'Setting up the files path'
    strRootFolder = ThisWorkbook.Path & "\"
    strSearchPath = strRootFolder & "AP Search Multiyear.xlsm"
    strSearchName = "AP Search Multiyear.xlsm"

    Workbooks.Open strSearchPath
    Workbooks(strSearchName).Activate

    'Disabling some options for better performance during the macro execution'
    Application.Calculation = xlCalculationManual
    Application.ScreenUpdating = False
    Application.DisplayAlerts = False

    '' Macro to update my Power Query script(s)
    Application.Run ("'" & strSearchName & "'!UpdatePowerQueries")

    ''https://excelguru.ca/2014/10/22/refresh-power-query-with-vba/
    'Screen update
    Application.Calculation = xlCalculationAutomatic
    Application.ScreenUpdating = True
    Application.DisplayAlerts = True

    ActiveSheet.PivotTables("PivotTable1").PivotCache.Refresh
    Workbooks(strSearchName).Save

    'SAVE AND CLOSE EXCEL PHASE'
    Workbooks(strSearchName).Close False

    ThisWorkbook.Activate
    ThisWorkbook.Save
    Application.Quit

    End Sub

  26. Hi everyone!

    First of all, thanks Ken for your article... piece of cake 😉

    I have a similar code than Alex, the same issue and (sadly) the same result when the queries are updated automatically by a macro ("Download did not complete"), but they are successfully updated if the process is run manually.

    I have tried some ways to solve this issue as:

    - Including the sentence "DoEvents" after the update process in the macro.
    - Updating each connection in a loop instead of using "RefreshAll".
    - Using the sentence Application.Wait("Time") after the update process with a several amount of seconds.

    I know (as you said Ken) Power Query doesn´t provide any VBA object model but, is there any way to retrieve the status of a query of Power Query?

    I have been thinking that if we can get the status, we can repeat the update process until the data download is complete.

    Thanks in advance.

  27. Unfortunately, not that I'm aware of, no. The thing is that in Excel 2013 we're not actually refreshing Power Query. We're actually refreshing the Excel connection, which happens to kick off Power Query. Being that Power Query is an addin, there is no flag that gets pushed back to Excel to say "I'm done".

    My feeling is that the only method we have to attempt to insert control here is to possibly flip the setting on the connection to turn off background refresh. My hypothesis here is that it SHOULD prevent Excel from moving on before refreshing the subsequent connections, but I could be wrong there. I haven't had time to fool around with it and come up with a reliable solution, unfortunately.

  28. @Alex, does this work?

    If lTest > 0 Then
    cn.BackgroundQuery = False
    cn.Refresh
    End If

    Regards

  29. Sorry, I solved the problem, but forget to point it to you.
    I added some lines, than enable refresh (by default disabled and after macros as well) and wait until queries done:
    On Error GoTo ErrorExit
    Application.OnKey "{Esc}", ""
    For Each cn In ThisWorkbook.Connections
    If Left(cn, 13) = "Power Query -" Then cn.EnableRefresh = True
    cn.Refresh
    Application.CalculateUntilAsyncQueriesDone
    Next cn

    Application.CalculateFullRebuild
    Application.CalculateUntilAsyncQueriesDone

    'Application.ScreenUpdating = True
    For Each cn In ThisWorkbook.Connections
    If Left(cn, 13) = "Power Query -" Then cn.EnableRefresh = False
    Next cn

  31. @Alex I've run into the same issue and implemented the solution you provided but still receive the same results (i.e. "Download did not complete."). My code is below; do you see any issues?

    For Each cn In TDR.Connections
    If Left(cn, 13) = "Power Query -" Then cn.EnableRefresh = True
    cn.Refresh
    Application.CalculateUntilAsyncQueriesDone
    Next cn

    Application.CalculateFullRebuild
    Application.CalculateUntilAsyncQueriesDone

    'Application.ScreenUpdating = True
    For Each cn In TDR.Connections
    If Left(cn, 13) = "Power Query -" Then cn.EnableRefresh = False
    Next cn

  32. Is it important which query got refreshed first in your VBA loop if we have a chain of dependent queries? (E.g. Q3 is merge of Q1 and Q2 ,Q6 is Q5 append Q4 and then Q7 is merge of Q3 and Q6)

  33. Hi Leonid,

    To be honest, after learning a bit more about this approach I think I'd actually just automate the Data --> Refresh All process wherever possible. At that point the query sequencing will take care of itself, and you'll also find that it performs better, especially if you have a larger number of queries. (We have discovered that the data model gets cached after each refresh operation, so if you run the macro above it would refresh and cache 7 times vs doing the cache only once with the data --> refresh all approach.)

  34. Not sure if others faced this issue but just updated Power Query and the Connection String of my Power Query tables changed to: provider=Microsoft.Mashup.OleDb.1

    Note the lowercase "provider" which you'll have to edit in the VBA code to compensate for this.

  35. Well that's more than a bit frustrating, isn't it? I'm curious, did this change for existing connections or just new ones?

    We can pretty easily fix this by modifying the lTest line as follows:

    lTest = InStr(1, LCase(cn.OLEDBConnection.Connection), "provider=microsoft.mashup.oledb.1")

    But we really shouldn't need to do this...

  36. Hey Matt,

    I just checked this, and I can't repro what you're seeing here. I just created a brand new Power Query using Excel 2016 (the very latest insider build), and it's working as expected, returning an upper case "P" for provider...

  37. Just troubleshooted the issue and believe I figured out the cause...

    I had created a file in which I used Power Query in Excel 2010. I then opened this file in Excel 2013. For whatever reason this permanently changes the existing connection's string from uppercase to lowercase "provider"

    I edited your VBA code (removing "provider" altogether) to make sure I never face this issue again:
    InStr(1, cn.OLEDBConnection.Connection, "Microsoft.Mashup.OleDb.1")

    Do you foresee any issues with me doing this?

  39. I have text file with 2 million data I need to import to excel by power query.. before doing this there are 5 columns which 2 columns are dependent . how can I import only 5 columns and get distinct values from that columns.... the table has duplicate rows. and there is no unique row to identify. then user will be given drop down list. the list will be populated by unique rows from those 5 columns and user can select the values from the list and extract the data..... Note: those 5 columns will be there in the text file.

  41. Hi Ken,

    Thank you for this article. It had been working refreshing the data. However, today it won't refresh at all. My process is I run the refresh, copy the result into another worksheet, and hide that worksheet. I made this dynamic date range. So if I enter a new date range and hit refresh, it should pull in new data. The date range with the refresh button is in another worksheet. When the refresh button is pressed, it will unhide the worksheet that has is used for the Power Query.

    Now, it won't refresh anymore when I hit the refresh button. I used the code that you provided. I just don't know what I did wrong or what happened.

    I am wondering if you know if I changed something within the spreadsheet without me knowing it? I updated to the new Power Query add-in.

    Thank you!

  42. James, did a workbook that has been working suddenly start failing, or did the method start failing when you applied it to a new workbook?

    I did just modify the VBA in the post a bit to add the ", vbTextCompare" to the end of the lTest line (inside the closing parenthesis). This should fix an issue if the case of the connection string changes. Beyond that, I'm not sure what would cause this, as we're not referring to any query names here.

    What version of Excel are you using here?

  43. Ken,
    Very good article. I used the macro in my VBA routine to refresh my tables. I am having a problem though that you may be able to help me with. After adding the routine and storing the file, upon reopening it, I get an error, saying that it is unable to create an ActiveX control. As it is, the file is lost to me. Do you know how to fix this?

  44. Hey Don,

    Honestly, no... the code I use in the article above doesn't reference an ActiveX control in any way. It's probably related to the button you used to trigger it, or some other control, but can't be certain without more info. This may be a good question to post in the forum at http://www.excelguru.ca/forums as there is something else going on here.

  45. Ken,
    I fixed the problem which was to re-load the Power Query add-in. It's working fine for the moment. I tried to load another workbook with tables and got the same error after I hit "enter" which told me it was the Power Query and not the sources. I figure the Active X control is found in Power Query itself, not the code. Thanks anyway for the help.

  46. Update:
    There is a conflict with Skype where some subroutine will not work. I need to not update Skype in order to make Power Query work. Otherwise I need to repair it first. I don't know if anyone else has this problem but this may help.

  48. Hi Ken,

    Just one question: would it possible to run a macro from another worksheet? I need to refresh several querys from a main worksheet to rule all the process.

    Thanks in advance.

  49. Sure. If you link the macro to a button, it doesn't matter which sheet the button lives on. I've built many solutions where there is a single sheet with multiple buttons to control the execution of each macro when needed.

  50. Hello,
    It realy useful. Thanks.
    Another issue with which i fighting.
    I'm working on few PC. So, I have problem with datesource for PQ.
    Maybe You have some solution how to change source automatically in PQ with VBA?

  51. Not with VBA in Power Query, no. VBA works outside Power Query.

    For dates the most likely thing you need to do is use change type -> using Locale to set it to the format of the source data. That will override your system settings.

  52. Hi, trying to find a solution to update the source string in Power query based on data entered in specific cell in my workbook on a separate tab. Do you have any ideas how to do this?

  54. Hey Ken,

    I've been looking all over the excel forums but there's nothing so far.
    I need to refresh a Power Map using a macro instead of the button itself so it would be like a real-time visualization of my data that is being imported from another source every 30sec.
    Do you think it is possible?

  56. I used your macro and it works. However, I'm trying to have the macro auto execute before closing the file. In other words, when the user saves the file and then clicks to close the file, before it actually close I want the macro to execute and save the file without any more intervention from the user. Come to think of it, it might be better if the macro execute on saving and not on close.

    Here how the code looks like now:
    Private Sub Workbook_UpdatePowerQueries()
    ' Macro to update my Power Query script(s)

    Dim lTest As Long, cn As WorkbookConnection
    On Error Resume Next
    For Each cn In ThisWorkbook.Connections
    lTest = InStr(1, cn.OLEDBConnection.Connection, "Provider=Microsoft.Mashup.OleDb.1", vbTextCompare)
    If Err.Number 0 Then
    Err.Clear
    Exit For
    End If
    If lTest > 0 Then cn.Refresh
    Next cn

    End Sub
    At this point the macro executes but it pops up a window warning that closing the file will cancel the query. Help!

  57. Hey Joao, I'm not sure that we can automate Power Map via VBA. Nothing comes up in the macro recorder, but to be fair I haven't dug into the object model too deeply.

  58. Hi Zuna,

    I believe you need to set the query to run as a background query, which will force it to load before moving on, then you could put a msgbox at the end. Something like this:

    Set cn = wb.Connections("Query - Name")
    With cn
    .OLEDBConnection.BackgroundQuery = False
    .Refresh
    .OLEDBConnection.BackgroundQuery = True
    End With
    MsgBox "Complete!"

  59. Ariel, try this version instead:

    Private Sub Workbook_UpdatePowerQueries()
    ' Macro to update my Power Query script(s)

    Dim lTest As Long, cn As WorkbookConnection
    On Error Resume Next
    For Each cn In ThisWorkbook.Connections
    lTest = InStr(1, cn.OLEDBConnection.Connection, "Provider=Microsoft.Mashup.OleDb.1", vbTextCompare)
    If Err.Number 0 Then
    Err.Clear
    Exit For
    End If
    If lTest > 0 Then
    cn.OLEDBConnection.BackgroundQuery = False
    cn.Refresh
    cn.OLEDBConnection.BackgroundQuery = True
    End If
    Next cn

    End Sub

  60. Ken, I tried the new version and it does not work. I get a popup window asking me if I want to save my changes, I click yes and it saves and closes the file. But when I open it again, the table is not refreshed.
    I know next to nothing about coding so I apologize. Do you know what might be malfunctioning here?

  64. In Excel Power Query, one can group multple queries into a folder or group. Is there a VBA code that can specify to refresh the group of queries using the Group folder name?

  67. Ken, How do you tell a macro to wait until the PQ query is complete before refreshing a pivot table? In my code below, the PT refreshed BEFORE the PQ is rereshed.

    With ThisWorkbook
    Set ptFees = shtptFees.PivotTables("PTFees")
    .Connections("Query - tblProducers").Refresh
    ptFees.PivotCache.Refresh
    End With

  68. Hi Mike,

    Try this and see if it helps:

    Set ptFees = shtptFees.PivotTables("PTFees")
    Set cn = ThisWorkbook.Connections("Query - tblProducers")

    With cn
    .OLEDBConnection.BackgroundQuery = False
    .Refresh
    .OLEDBConnection.BackgroundQuery = True
    End With

    ptFees.PivotCache.Refresh

  70. Excellent, worked from the first try,

    Thank you Ken, you are awesome and help was just on time!

    Have a great day
    Anna

  71. Hi, Ken! I remember I once saw a post from you somewhere about the VBA code to define the order in which we want Queries to refresh. Can't find it. Would you by chance know where that solution could have been posted?

  72. Hi Celia, I honestly can't remember either, but...

    By default, Queries with load destinations refresh in alphabetical order (refreshing any child queries along the way.) If you need to use VBA to refresh them in a specific order, then I'd just record a macro to refresh one, then copy the line of code and tweak it for the other queries.

  73. Hi Ken,

    In order to avoid the inexplicable glitches that can occur when refreshing queries with VBA (inexplicable to me anyway), my approach (in 2016) has been to create queries with VBA to generate the results then delete all queries at the end of the process. In other words, my practice is to delete all queries from the workbook before closing and then recreate them with VBA either on user-demand or automatically when the workbook is opened again.

    I never figured out what caused the intermittent, seemingly random errors with the VBA-driven refresh...I just know that moving to the deletion/recreation technique solved that problem for me in every case.

    Our enterprise is a mix of 2013 and 2016. I'm trying to develop a Power Query solution that works in both environments, detecting the version and then running different code based on the version. The problem is that I can't figure out how to apply the delete/create technique in 2013. Again, I don't want to leave queries in the workbook to be refreshed by VBA as I've had so much trouble with that in the past. Is there any way to apply the delete/recreate technique in 2013?

  74. Hi Michael,

    I'll be honest, I've never had any glitches when using VBA to refresh queries. I build solutions frequently to do exactly this, link it to a button for end users and then have a way to control which queries get refreshed, and in what order if that is an issue. I'm curious why you're having issues, as that isn't normal.

    With regards to 2013, the issue you have is that Power Query is an add-in, so there is no object model to delete or create queries. Refresh is different, as a macro can be recorded to refresh the data connection.

  75. I've been meaning to post this for a while, but, you know...
    Based on the work posted by Ken and others, I built a function to update a query by its name. It has worked in Excel 2010 and Excel 365 very reliably for queries to Excel, CSV, SQL and Access, as well as internal queries from tables in the same excel file.

    The "QueryName" value in the calling function is the natural Power Query name of the query.
    Hope that this helps anyone struggling with VBA updates for Power Query.

    Calling function:
    ….
    Dim Result as string
    Result = updatePQ("QueryName")
    If Result Like "*Error*" Then Err.Raise (1001) 'calls error handler
    ….

    Update Function:
    Option Explicit
    Function updatePQ(nmQuery As String) As String
    Dim qName As String
    Dim ix As Long
    Const qPrefix As String = "Query - "

    On Error GoTo Err01
    ix = InStr(nmQuery, qPrefix)
    qName = nmQuery
    If ix < 1 Then qName = qPrefix & nmQuery

    With ActiveWorkbook.Connections(qName).OLEDBConnection
    .BackgroundQuery = False
    .MaintainConnection = False
    .Refresh
    End With
    updatePQ = "OK"
    XIT:
    Exit Function
    Err01:
    updatePQ = "Error - PQ Refresh: " & nmQuery
    Resume XIT
    Resume
    End Function

  76. It appears that if your query is added to the Data model in Excel, that the VBA code will "wait" until the query completes before executing the next line of vba code. Can anyone else confirm this?

  77. Hi Mike,

    I just did a VERY quick test on this, and from what I can see, yes. The code waits for the model to complete it's operations before executing the next line of code.

  78. Hi Ken , Thanks a lot finally I find the macro code that helps me to refresh the power Query. Just now I need your help what is the code of macro after refresh power query refresh pivot table

  79. Hi Pegah,

    If you turn on the macro recorder, you can right click your Pivot Table choose Refresh, and you'll have the code. You may also want to just record a macro to do the Data -> Refresh All action as that will refresh the queries and all PivotTables.

  80. Thanks Ken. I write this code the end of your code but I must run the macro tow times then the pivot table will be updated. would you please guide me which part is wrong.
    Sub Run()
    '
    ' Run Macro
    '

    Dim lTest As Long, cn As WorkbookConnection, pt As pivotTable
    On Error Resume Next
    For Each cn In ThisWorkbook.Connections
    lTest = InStr(1, cn.OLEDBConnection.Connection, "Provider=Microsoft.Mashup.OleDb.1", vbTextCompare)
    If Err.Number 0 Then
    Err.Clear
    Exit For
    End If
    If lTest > 0 Then cn.Refresh

    Next cn
    For Each pt In ActiveSheet.PivotTables

    Select Case pt.Name

    Case "PivotTable1"

    pt.RefreshTable

    Case Else

    End Select

    Next pt
    End Sub

  81. Hi Pegah,

    Try updating your lTest/next cn lines with the following:

    If lTest > 0 Then
    cn.OLEDBConnection.BackgroundQuery = False
    cn.Refresh
    cn.OLEDBConnection.BackgroundQuery = True
    End If

  83. Hi Ken,
    I’m having the same problem as Pegah in needing to run my VBA code twice to achieve the result I am expecting. Adding BackgroundQuery statements as follows:
    With cn
    .OLEDBConnection.BackgroundQuery = False
    DoEvents
    .Refresh
    DoEvents
    .OLEDBConnection.BackgroundQuery = True
    End With

    is not successful in forcing the table to be updated in time for the next VBA steps which use the information from the table. The result after the VBA code completes is that the PowerQuery has updated its table correctly, but the subsequent tables that are populated from the query data are missing information, because it was updated too late for the next step.

    I have tried DoEvents and Application.Wait Now + TimeValue("00:00:ss"), but these merely slow the process without achieving the desired result. I've also tried creating a new sub to run the code twice, but this falls over at the second running of .OLEDBConnection.BackgroundQuery = False with an Error 1004 "Microsoft Excel is refreshing some data. Please try again later"

    It's very hard to debug this problem, because the mere fact of pausing while stepping through means that I never encounter an error because the query catches up.

    I’d really appreciate your thoughts, please.

  84. Geoff & Pegah,

    If you scroll up the comments to Jan 11 2016, Alex posted a solution that has some extra components in it about calculating AsyncQueries and a FullRebuild. Can you see if slotting some of those in fixes the issue?

  86. Hi Ken,
    Is there a way to have a secondary query refresh when a cell from the primary query is clicked on? Example, if I click on the word 'Cat', I would like it pass that as a parameter to my secondary query to retrieve additional details into a separate worksheet. I should note I already have a parameter table for my primary worksheet and there's no desire to that the primary refresh automatically.

    Any ideas would be greatly appreciated.
    Thanks

  87. Hi Christine,
    The only way I can think of that you could do this would be with VBA's Worksheet_SelectionChange event. You'd want to set up a query specifically for this, then use the event to update the query, then refresh it. No idea how performant it would be, but I think it would work.

  88. To add my two cents, I would do exactly what Ken suggests, with the caveat that I would read the table back into PQ from the sheet as the source for the secondary query rather than using the primary query as a direct source, since that would cause the primary query to refresh first.

  89. Hi Ken and AlexJ,
    Yes, the secondary query is not sourced from the primary query. I would just like to use a value from the primary query as a parameter value to call the secondary query. Does this change your suggestion?

  90. Hi Christie,

    If you go the VBA route, it would work. Each query (in Excel 2016 and higher) has a .Formula property which contains the M code for the Power Query. (What you see in Power Query's advanced editor.) If the SelectionChange event were to update the query's .Formula to a new M code formula, then refresh it, that should do the trick.

  91. Hi Ken -

    Thanks so much for this tip, it is very helpful. QQ though - how do we know that the power queries are refreshed in the right order? For example, if Query 3 has dependency on Query 2, and 2 on Query 1, we want them refreshed in the correct order. As per your code above, it seems to just refresh them in a random order. Please let me know if i am missing something. Thanks!

  92. Hi Sid,

    If you refresh Query 3, it will automatically refresh the child objects Query 1 and Query 2, and there is no way to disable that. Assume though, that you wanted to refresh Query1, land it to a table, then add some info to another column. Then you wanted to merge that table with Query 2, you would want to refresh Query1 out of sync, so it would make sense to do this.

    With regards to the query chain, barring dependencies, it refreshes in alpha order I believe. There is no dependency information available via the VBA object model, but if you want an easy way to get it, consider trying https://excelguru.ca/monkeytools as we do provide that in the QuerySleuth form.

  93. Hi Ken,

    I want to refresh connections only queries (power queries) using vba and read the record count from the connection. If record count is more than 0, I want to publish result in separate worksheet. Not sure, if this is possible. please advise
    On another note : do you know what is best way to get only results where record count is more than 0 in excel in place of having all the sql linked to separate worksheet for results.

  94. Hi Amit,

    Unfortunately, the only way to get a row count is to do the full refresh and then read it from the resulting table. We have no ability to read the previews (which aren't full data sets anyway.)

  95. im stomped i have a workbook where if i update queries manually they update fine and within a minute each. But if i use update all, they never finish. Any ideas?

  96. Ken,
    I use a button and apply the following code and it refreshes all queries in the workbook. What would be the advantage of your code over what I am currently using.

    Sub Macro4()
    ActiveWorkbook.RefreshAll
    End Sub

    Alan

  97. Hi Ken,

    Would you please confirm for me that the Sub Run() macro from Aug 19, 2015 runs all queries in the order of query dependencies - similar to how Refresh All would work?

    I have a number of queries that are dependent on outputs from other queries, and am trying to determine whether this script would be useful for including in a For loop to update across multiple states.

    I am able to run this script in the For loop flawlessly, but once again, am just looking for confirmation that the refreshing of all queries runs in the order associated with query dependencies.

    Thank you,

    Geo

  98. Hi Geo,

    No... the native Power Query object model has zero idea of any dependencies. This code allows refreshing queries in the order they appear in the queries pane. If multiple queries share children, those child queries will get refreshed as each parent is updated. If you want to have all queries refresh while observing the proper dependency chain, then you should use the code that Alan posted immediately above.

    The reason you'd use this code is to force queries to update in the specified order, despite the fact that it may actually be slower to do so.

  99. Pingback: Calculate Hours Worked in Excel with Power Query

  100. Pingback: Workbook Privacy Settings

  101. That's great Ken. But what if I need the queries in my workbook to update in a specific order? Can I "force" that to happen? Let's say these are my queries (in the order I want them to refresh.

    data
    pending
    updates
    revised
    finals

  102. I would record a macro, then refresh each individually to get the syntax. Just be aware that this will (usually) not be as efficient as doing a refresh all, but in the case were a specific order is needed (or a partial refresh is desired) it works well.

    Here's what I got recording the refresh of "MyQuery" and then a Refresh All operation:
    ActiveWorkbook.Connections("Query - MyQuery").Refresh
    ActiveWorkbook.RefreshAll

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Latest Posts

Copyright © 2022-present Excelguru Consulting Inc. All rights reserved.