Tableau Server – all about the… Backgrounder

Hello everyone.

You may know me as a Tableau Centre of Excellence manager. That can involve a lot of paperclip pushing skills, with the real work being done by my excellent team (thx @jakesviz & The Information Lab). But I do try and get down and dirty with my lovely Tableau Server environment to keep my skills fresh. Obviously I don’t mess with it – @jakesviz gets pretty protective about his Server.

This series of posts is my attempt to shed some light on the internals of Server. Note there are many more experts in this field than me (Craig Bloodworth, Mark Jackson, Jen Vaughan, Tamas Foldi, Mike Roberts, Angie Greenhaw – to name but a few) so please do comment if anything is incorrect here. Maybe you guys could help me evolve this post?

What is the backgrounder?

The backgrounder is a process that runs as part of the Tableau Server application. As the name suggests, it handles background tasks such as refreshing extracts, running subscriptions and also processes tasks initiated from tabcmd.

Here are the backgrounder processes. The .exe file and the .war file. The WAR file is a Web Application Archive, and contains all the necessary components and resources needed for a Web Application such as Backgrounder.

On a clustered environment you’ll find these files in D:\Program Files\Tableau\Tableau Server\worker.1\bin (may vary slightly with your installation).

28-01-2016 14-07-52

Other files related to the backgrounder.

Template (.templ) files – These files TBD

28-01-2016 14-13-09

There are also a few .rb files in D:\Program Files\Tableau\Tableau Server\worker.1\tabmigrate\db\migrate.

And we also have a .properties file which contains all the config entries relevant to the backgrounder. It also has almost all of the other stuff that you’d find in the main workgroup.yml file which is odd. I’d  have expected it to be just the backgrounder config.

backprops

Here is the location of the backgrounder log files

backlogs

Here are 2 instances of backgrounder.exe running on my server (from Task Manager).

28-01-2016 14-01-08

 

Can I mess with it?

Backgrounder can be configured. There are several settings present in both workgroup.yml and backgrounder.properties. Workgroup.yml is the master config file, and it populates the backgrounder.properties (and other .properties files) when a ‘tabadmin configure’ is run.

I don’t know what all of these do (yet) and the only one I’ve ever edited is ‘backgrounder.extra_timeout_in_seconds‘ which sets the max time in seconds that a backgrounder session can run for. Tableau kills off the session if this threshold is reached. Useful for forcing users to optimise their extract times!

I also pay attention to the ‘backgrounder.vmopts‘ parameter, as this defines the size of the java heap space for this component. All components have a vmopts setting and I’ve had to increase them on occasion due to out of memory problems.

You may also want to change the ‘backgrounder.log.level‘ if you need more debug info, although Tableau logs are chatty enough for me.

If there’s a golden parameter in this lot that you get value from then let me know in the comments.

backgrounder.deploy.dir: D:/Program Files/Tableau/Tableau Server/data/tabsvc/backgrounder
backgrounder.external_cache.concurrency_limit: 10
backgrounder.external_cache.enabled: true
backgrounder.external_cache.num_connections: 1
backgrounder.external_native_query_cache_disable: true
backgrounder.extra_timeout_in_seconds: 1800
backgrounder.failure_threshold_for_run_prevention: -1
backgrounder.jdbc.wg.connections: 8
backgrounder.jdbc.wg.idle_connections: 4
backgrounder.log.dir: D:/Program Files/Tableau/Tableau Server/data/tabsvc/logs/backgrounder
backgrounder.log.level: info
backgrounder.native_api.log.level: info
backgrounder.out_of_date_schedule_minutes: 240
backgrounder.ping_dataengine.millis_to_wait: 5000
backgrounder.ping_dataengine.num_retries: 24
backgrounder.ping_services.num_retries: 10000000
backgrounder.ping_services.time_to_wait: 5000
backgrounder.purge_directories.directories: ""
backgrounder.querylimit: 7200
backgrounder.restart_interval_in_minutes: 480
backgrounder.restrict_serial_collections_to_site_level: true
backgrounder.search_index_verification.enabled: true
backgrounder.sheet_image_api.max_age: 240
backgrounder.sleep_interval: 10
backgrounder.sort_jobs_by_run_time_history_observable_hours: -1
backgrounder.sort_jobs_by_type_schedule_boundary_heuristics_milliSeconds: -1
backgrounder.timeout_tasks: refresh_extracts, increment_extracts, subscription_notify, single_subscription_notify
backgrounder.tomcat.threads: 4
backgrounder.urlprefix: backgrounder
backgrounder.vmopts: -XX:+UseConcMarkSweepGC -Xmx512m

Note that Tableau Support don’t like you to edit config files manually, they recommend that you use the tabadmin set commands to change any parameters. They might have to change that recommendation when we see Tableau Server on Linux.

For more about .templ, Ruby & properties files check out this from Tamas Foldi.

 

What are the problems with the backgrounder?

Here are some of the things that can be problematic with the backgrounder.

  • Single Threaded – This means the backgrounder process can only run one thread at a time, a thread being a set of executable instructions that a process can perform. The upshot of this is that your backgrounder works through a queue of tasks one-by-one.
  • Latency – Due to the single threaded nature of backgrounder, you may see delays or ‘latency’. For example, if you have one backgrounder, and 2 tasks for it to perform at 2am, then task 2 will have to wait until task 1 has finished. If task 1 takes an hour then task 2 won’t start until 3am. This can be annoying for users that expect their data to be refreshed by a certain time.
  • Resource Intensive – The backgrounder can consume a significant amount of processing power (CPU) and input / output (I/O) on your server. This is dependent on the type of task it is performing. It’s not uncommon to see a backgrounder node consuming 100% CPU.
  • Other Stuff – The backgrounder process also does other stuff on Tableau Server that isn’t concerned with extract refreshes. For example – reaping extracts, checking disk space, synching Active Directory groups, rebuilding the search index etc. Bear that in mind when building your system. In reality these tasks don’t take up too much resource but they do take some and you should be aware.

 

Isolating the backgrounders

A common configuration in a clustered environment is to dedicate one of your worker nodes to the backgrounder processes. This means you can dial up the number of backgrounders and let them do their stuff without worrying about any impact on other processes. This is one of the most common performance recommendations from Tableau support.

You can also get a lot of info out of the Postgres DB relating to backgrounder usage and performance. Nelson Davis has posted a guide to getting started here.

 

Improvements I’d like to see

Ok so here are a bunch of improvements I’d like to see to the topic of backgrounders and extract management. I know some of this will drop in upcoming releases and some of these problems have been solved with custom solutions at some customer sites.

Alerting – Tableau Server doesn’t alert (email / IM / SMS) when a task fails. This means you’ll need to set up external monitoring to detect issues. I know Tableau are on this though so expect to see it in an upcoming version. Some people in the community have also coded their own solutions to this problem but it really should be native functionality.

Control per site – We segregate our dev / test / prod user environments using sites , all on the same server. We run 8 backgrounder processes on that server, which are shared across the tasks on all sites. As an administrator I’d really like to be able to bind backgrounder processes to specific sites. For example, 2 backgrounders on each of dev & test sites, then the other 4 dedicated to the production site. That would ensure production tasks always have enough resource to be able to execute on time.

Control per process – I’d like to be able to stop / pause / mess with individual backgrounder processes easily. It is possible – see this from Toby Erkson, but it would be good to have this as part of an administrator console or something.

Control per type / size / pattern of extracts – It would be good if I could dedicate specific backgrounder processes to particular extracts based on their characteristics. In particular I’d like to allocate one backgrounder to all the extracts that take less than 1 minute to complete. Or even use this to reward users that show diligence with their extract management by dynamically prioritizing incremental refreshes or extracts that have a low failure rate.

Better metrics – I’d like to see exactly how much CPU is taken up by a particular backgrounder process or task / schedule or per project. This would be useful for chargeback.

Dynamic reprioritisation – I love all my users. But in particular the ones that take good care of their extract refreshes. I’d like Tableau to be able to dynamically increase the priority of tasks that complete quickly, are incremental and that have a low failure rate. The message being, if you want your stuff to get the best slice of available resource then help us out with best practice.65

Disable run now – We’ve had some issues with the “run now” option that allows users to kick off an extract refresh on-demand using the UI. In particular we’ve seen some trigger-happy users bring our server down by hammering on the run now option multiple times. I’d like to disable that or maybe throttle it somehow.

Better guidelines from Tableau – The documentation from Tableau isn’t great in this area.

05-05-2016 13-09-11

I have an 8 core 128GB server and  run 8 backgrounders with no capacity issues. And I know of other organisations running way more than that. According to this doc I should be running between 2-4. That would be some serious under-utilization of the server. I’d like to see some clearer recommendations, maybe taking into account the variety of use cases that I’ve seen in other big enterprise deployments.

 

OK that’s all for now. This post could have been more detailed but I figure that I’ll get some valuable inputs from the community that will help me expand it. Actually in the time I was writing this, Mike Roberts was doing the same – check his post for some excellent info.

Cheers, Paul

 

Advertisements

2 minutes with… Carl Allchin of The Information Lab

2 mins with title2

Hello folks. Time for a new 2 minutes with… I’m delighted to welcome Carl Allchin to VN towers. Carl spent 6 months with my team helping us build our Tableau service. He’s got great Tableau Desktop skills and has a natural flair for helping to train users.

Without further ado..

VN: So who are you then and what do you do?

DSC_1535 (1)CA: I am Carl Allchin and I work for The Information Lab (Tableau’s Partner of the Year in EMEA for 2013) as a Tableau and Alteryx consultant and blog about Tableau on datajedininja.blogspot.com.

VN: Tell me about your org

CA: The Information Lab is a great company. Started by Tom Brown a few years ago, they are now a band of Tableau and Alteryx gurus in Europe who have partnered with those two software companies to help everyone get access to their data and make best use of it. I’ve  recently joined from Barclays where I was an internal Tableau consultant (Data Ninja) but made the jump to The Information Lab so I could work across multiple industries and support more people get the most out of their data.

VN: How do you personally use Tableau?

CA: I am a Tableau-holic. I use Tableau for my day job to help our clients out with specific challenges they are facing. I am also a Tableau trainer so I get to teach people how to get the most out of Tableau Desktop. I get a massive buzz from seeing people have the same revolutionary moment that I had two years ago when I was taught by Tom to use Tableau in a course at Barclays. I also use Tableau in the evenings to analyse basketball data to understand the game I love more and to communicate with fans worldwide about the sport.

VN: Tell me more about your fascination with Tableau and sports data.

Carl Lay-upCA: Ever since meeting Peter Gilks at Barclays and talking data visualisation and Tableau, my career (and personal life) has gone on a fantastic journey. Being focused on data visualisation and Tableau at Barclays was great but Peter and I were also basketball nuts so we started to translate our day jobs on to basketball data so we could understand more about the game than ever before. As our Tableau skills and knowledge of the game grew, we started to use one to learn more about the other. For example, I wanted to learn how to use background images in Tableau so I create my interactive shot chart (http://public.tableausoftware.com/shared/GRGB8BZB9?:display_count=no) and Peter explored visualisation techniques like his ‘Ball Code’ (http://paintbynumbersblog.blogspot.co.uk/2014/04/how-i-built-ballcode.html).

This has led to me landing some really exciting opportunities, like being invited by Ben Jones to present at the Tableau Public session at the Tableau Conference in Seattle and talk just about basketball. That is pretty unique for a Brit to get the chance to do that in the US!

VN: What does the Tableau community mean to you and who do you learn from?

The Tableau community is simply amazing and one of the reasons why I love doing what I do. The fact that so many highly skilled, intelligent people are willing to help in any problem you come across is great. I could name so many people who I learn from but I have to give special mention to The Information Lab team (http://www.theinformationlab.co.uk/team/) as they were great to me before I even joined them, Lee Mooney for his database and analytics skills he still shares with me, Jewel Loree and Ben Jones of Tableau Public fame and all the Zens who always make time. That Viz Ninja fella is also pretty handy for some Enterprise Server tips too.

VN: So why are you a Data Jedi Ninja?

CA: At Barclays, Peter, Lee and I were the team that people came to when they wanted an ultimate approach than the traditional IT BI report so we basically hid away creating great stuff and only came out of hiding to deliver some great tools to the business before hiding away again and working on the next piece of data magic. We actually got Barclays to accept Data Ninja as our job title which was quite an achievement for a bank but as that Twitter account had already been claimed then I had to add an additional level.

VN: Could you give me an interesting non-work fact about yourself?

I’ve recently been on a seven week holiday to Australia to complete detox from all things data. Unfortunately, my girlfriend is did let me take my laptop so I just hope my Tableau withdrawal symptoms are not too bad.

That’s it for this one. If you ever get the chance to attend one of Carl’s training courses then do take it all in. He’s one of the best.

Cheers, Paul

 

2 minutes with… Nelson Davis of Slalom Consulting

2 mins with title2

 

Welcome back to what is now the second best BI interview series out there! Sob sob. Hey I can handle that Dan stole our idea, mainly because his Interworks interview series is so damn good!

But that doesn’t stop us bringing you some top guests. And this time it’s new Zen Master Nelson Davis of Slalom

So who are you then and what do you do?

Nelson ProfileI am Nelson Davis – a good southern gentleman born and raised in the beautiful city of Atlanta, GA. Just over a year ago I join Slalom Consulting to focus on Tableau and since then I’ve had some incredible opportunities to do amazing work. As of a few days ago I became a Solution Principal of Data Visualization for the Information Management and Analytics practice for the Atlanta office. I use Tableau in almost every aspect of my day to day work.

Tell me about your organization

Slalom is a consultant’s dream place to work. Rather than getting on a plane each week and traveling across the country, Slalom employs a local model, serving clients in each of 15 our markets (newest one is in London – and looking for amazing people). Because of this, we’re invested in long term relationships with our clients – in Atlanta this means I’ve had the opportunity to do work for the likes of Home Depot, Coca-Cola, Delta, UPS, AT&T and Cox Communications. It also means that we get to be invested in the community – over the course of the year a group called Slalom Cares Atlanta organizes Backpack drives for school kids in need, Holiday Gift Drives, Can Food Drives, and more. And the people of Slalom are amazing themselves. It became clear to me when I first walked in the door that Slalom goes out to find the brightest people in the marketplace, puts them together to work on the most challenging projects, and gives them free reign to own the outcomes – which are often time amazing.

Last thing I’ll say is that we’re Tableau’s North American Alliance Partner of the Year for 2013, and we have an internal Tableau Community that is very collaborative and working to push the envelope in every direction when it comes to Tableau.

How did you start using Tableau? And how are you using it now?

My path to Tableau is a bit odd. In school I studied Civil Engineering with a focus on Transportation (I like connecting things). After graduation, my wife and I did a year of mission work in Mexico, only to return home in the Spring of 2009 to near double digit unemployment. With my Master degree I landed a systems analysis internship (yay $15/hr!) and got to play with data for the first time. Many months later I landed my first Transportation Engineering job – and almost immediately realized that data was more fun. After a little over two years as an engineer in a two person office, my boss was offered another job and he parleyed a spot for me – they were looking for a transportation data analyst.

After a few months of Excel (the gateway drug of data nerds everywhere) and some cool GIS stuff in Google Earth, I was ask to find a way to create ‘dashboards’ with ‘live data’. I said sure and promptly Googled ‘what the heck is a dashboard?’ and Tableau popped up (v6.0). That afternoon I made my first dashboard, and things just kinda took off. I got really involved in the Atlanta Tableau Users Group and started raising my hand when they asked for presenters. Within a few months I was using Tableau on projects coast to coast, and traveling to support them. I started to realize the amazing potential of Tableau and how to hack it to push the limits. When I showed off Google Street View integration in a Tableau Dashboard to the TCC13 speaker selection people, I got an invite. TCC13 opened my eyes to the amazing Tableau Community and the transition over to Slalom helped me to take my Tableau work to the next level. I’ve done everything from Black Friday Analytics, to Social Media Reporting, to Online Merchandising, to Logistics Networking and much more – all in Tableau. It’s been a lot of fun.

Who do you learn from in the Tableau community?

There are a number of people in the community that are putting out amazing things. Like many others I point at Joe Mako, Jonathan Drummey and Andy Kriebel as kind of the Godfathers of the community.

However, I find that the stuff the most inspiring, technical, and what I want to emulate comes from the likes of Mark Jackson, Ryan Robitaille, Ben Sullins, and Russell Christopher. These guys are the boundary pushers, extending what’s possible in Tableau with some crazy integrations, customizations and making Tableau not look like Tableau. My goal in my work is to make the tool disappear – allowing the user to focus only on the analysis.

You do tons of work with the Atlanta Tableau Users Group. What makes you so keen to help others?

I’ve presented at ATUG four or five times and will do so again in December (email me if you want to join the webex!). I think my mom, who taught for over 30 years, instilled a love of teaching in me. The best part of my job is working with clients and watching for the light bulb moments, when everything just clicks in to place and they see or understand something for the first time. That’s what gets me out of bed in the morning – helping people connect the dots. This same thing is at the core of what Tableau’s all about – the belief that seeing the data visually will fundamentally change the user’s understanding.

So I guess you could say Tableau and I are kinda perfect for each other. I’m also just passionate about making complicated things simpler, putting power in to the hands of more people and spreading the word that there is a better way out there – and I think the secret is getting out.

You got the honour of being named a Zen Master this year. What does that mean to you?

Wow. Great question. Earlier this year Slalom did a professional workshop and asked about a career ‘bucket list’. One of the three things I wrote down was that one day someday I hoped I might be named a Tableau Zen Master. I never thought there was a chance it would happen this year, or if it might ever happen. It’s not something you can try to do, there’s no list of check boxes anywhere. The Tableau Community is full of bright and talented people, but you want to talk about some of the most amazing, cream of the crop people on the planet – that has long been my opinion of the Zen Masters. They put out the best work and they give of themselves relentlessly to help others in the community.

Finding out that I would become a part of this group was both amazing and terrifying, honoring and humbling, all at the same time. There were certainly moments where I thought to myself ‘but I can’t do table calcs like Joe and Jonathan, Alan’s mapping is way better than anything a can do, Kelly and Anya do such better designs than me….’ And yet I’ve come to realize that while all of that is true (and it is), there’s a place for me, and the passion that I bring, in this group. It is humbling in every way, and I believe there’s a sense of responsibility to the community and to Tableau that I will work hard to honor (though I’ve promised my wife, I’m not doing 30 for 30 ever again).

Could you give me an interesting non-work fact about yourself?

Nelson Profile2I’m passionate about global missions and photography. I’ll be taking a big trip back down to Mexico at the end of March 2015 to help build a house for a family that lives in conditions that don’t really exist in our first world lives. I also (sorta) maintain a photography website – nelsondavisphotography.com – that

I enjoy sharing some fun work from the past. But I’m most passionate about my family – two young boys and my awesome supportive wife make it such that there’s never a dull moment.

Okay that’s it. Tune in next time for more guests before Dan steals them…

2 minutes with… Mark Jackson of Piedmont Healthcare

2 mins with title2

 

Ok we are back back back! And it has recently come to my attention that there is now a competing interview series. Well I guess imitation is the best form of flattery so I won’t send the elite Ninjas around just yet. Besides, it’s a great read so I’d recommend you check it out!

But we’ve got a blinding guest this month, none other than one of my absolute Tableau heroes – Mark Jackson. Without further ado…

VN: So who are you then and what do you do?

100_0670

Nice hats!

My name is Mark Jackson. I go by @ugamarkj on Twitter and have a blog at http://ugamarkj.blogspot.com. I run the Tableau BI program at Piedmont Healthcare in Atlanta, GA. My mission is to create an army of viz ninjas (Ahem! they all work for me actually – VN) and turn them loose to do awesome things with our data. So I spend most of my time educating others and modeling data sources for them to work with. I now have a team of two senior BI developers that I’m able to transition some of that work to. So I’m able to spend more time these days thinking strategically and exploring new technologies like NOSQL graph databases to help answer really complex relational or pattern based questions.

 

VN: Tell me about your org

Piedmont Healthcare is a five-hospital system. We have 400 employed medical staff members, and we have approximately 1,200 affiliate physicians with more than 100 physician practices across North Georgia. In total, we have around 10k FTEs that work for the heath system. Piedmont has some of the best medical providers in the world, and I wouldn’t go anywhere else for treatment. We had our second child at Piedmont and it was a great experience. So I’m honored to be a part of this organization and play my small part in improving the care we provide to our patients.

VN: How do you personally use Tableau?

These days I mainly use Tableau to evangelize Tableau and the science of data visualization. Tableau is the best tool on the market to help people rapidly see and understand their data. No other tool even comes close at the moment. Sure other tools can build beautiful interactive dashboards. But Tableau is leaps and bounds faster at doing it because it lets you iteratively paint with data until you arrive at an insightful view. And because you are painting with data using VizQL instead of plugging data into prescriptive dialog boxes to create specific visualizations, it is vastly more flexible and lets you be super creative. The fact that Joshua Milligan created tic-tac-toe and blackjack with artificial intelligence in Tableau proves that point. Try doing that in QlikView.

 

VN: You’ve always been one of the community interested in the IT / enterprise side of Tableau. How come?

It is because I run the BI program for large healthcare enterprise. My role exists within Financial Planning and Analysis where we have a shadow IT program. So we are responsible for a large portion of activities that would traditionally be within IT. This includes data governance, master data management, security, streamlining data movement, optimizing queries, transforming data, ensuring application up-time, troubleshooting software issues, functioning as a help desk…and the list goes on. If I don’t get the infrastructure right and instill confidence in the reliability of the system, I’m doomed. I started out at Piedmont as a customer of data analytics services in my role as Manager of Business Development and the Project Management Office for our Heart Institute. When I moved into my current role, my goal was to enable everyone to easily do the things I was able to do and bypass all the mistakes I’ve learned from along the way. To do that required setting up a sound architecture to remove a lot of the complexity required to do analytics work.

VN: What does the Tableau community mean to you and who do you learn from?

I echo what Steve Wexler recently wrote in a blog series about the Tableau community. This has to be the best software focused community on the planet. I’m tracking more than 80 blogs that post about Tableau at http://ugamarkj.tumblr.com. That is crazy and just shows how much excitement there is about this tool. The best thing about the community though is that it has managed to be dominated by people who are humble and super giving of their time. I’ve received so much help and inspiration from the community that it is impossible to name names without leaving important people out. Folks on the Tableau forum played a big role in my early development and more recently I’ve connected with a lot of people on Twitter. Many of these people I’ve met in the real world at the Tableau Conferences. They are all wonderful people that I feel privileged to know. I’m an active member of the community because the culture is amazing and I enjoying sharing what I learn with people that I consider to be my friends.

 

VN: Could you give me an interesting non-work fact about yourself?

DSC_0013People that befriend me on Facebook or meet me in real life will learn that I love to talk about the things that they always tell you not to talk about: politics and religion. I love to think about things philosophically measure actions against principles. With these topics (and any topic really), it is important to stay humble and be willing to look at things from someone else’s perspective. I can’t tell you how many times I’ve changed my mind on things because someone else challenged my point of view. I’m incredibly grateful for those people. My wife Leigh is one of those people that helps me think though these things (God bless her for putting up with me). I’m also a father of two kids (Ansley-4 and David-2). They keep me on my toes and are always good for a laugh.

That’s great thanks a lot Mark. And thanks for all the support with the infra related aspects of Tableau.

Ok see you next time folks – VN

How to initiate IM / Skype / Chat / LastFM session from a Tableau dashboard

Top Tips

Hello everyone. Before I start this I have to admit I nicked the idea completely from Dave Hart of Interworks, who came up with it when he was working with us. But he doesn’t blog or tweet so that leaves me to claim all the glory… Actually I’m just passing on his superior knowledge..

Many of you probably use mailto: actions in your Tableau dashboards, to kick off a mail to someone that might be highlighted in a viz etc. And you might also have some sort of internal IM system like MS Lync or something like that.

Well you can just as easily initiate a chat session to one or multiple recipients using Tableau.

This uses the Session Initiation Protocol (SIP) to open the session in whatever IM client is in use. You just need to open Tableau Server like this

tabadmin set vizqlserver.url_scheme_whitelist sip
tabadmin set vizqlserver.url_scheme_whitelist im
tabadmin restart

sip: allows Lync to lookup people using their email address
im: tells Lync to open it as a group chat

This is an example of an URI Scheme. There are dozens of others, such as callto: which tells mobile phones to call as well as facetime: and of course mailto:. 

Once you’ve opened the server up you can then create a Tableau action to fire off the command. Note that if you try and IM yourself it will usually default to email.

im:<sip:kelly.smith@company.com>
im:<sip:kelly.smith@company.com><sip:james.anderson@company.com>
im:<sip:kelly.smith@company.com><sip:james.anderson@company.com><sip:rob.jones@company.com>

When it comes to the action Tableau won’t let you use multiple items without specifying a delimiter so what you do is use “><” as the delimiter and then put [url encoded] “<” and “>” around the data field.

Anyway I posted this on the Tableau Forum and there’s a workbook attached to that post so you should be able to download it and see how you get on. I reckon there’s some potentially way cool stuff possible using URI schemes. There’s also lastfm:, spotify: and skype: amongst them – so much fun to be had here I think.

Forum post: http://community.tableausoftware.com/thread/155429

Regards, Paul

Talkin’ bout a Revolution…

Hello, I trust you’re all ok,

There’s something stressing me about Tableau. It might not be the most obvious but I’ll have a go at describing it anyway.

See I demo Tableau Server all the time. Like daily. To some pretty senior people that I really want to impress. These demos often involve clicking around the server views to show some of the user content. Trouble is some of my user content isn’t that great. It can be badly designed and slow to load – that’s a separate issue that my team is working on.

So I click on the view and then, there it is…..

30x30REV

Spin, spin, spin. Will it be 2 seconds or 20? Will it even load at all? The room falls quiet as all eyes settle on the spinning circle. The audience is almost hypnotised. The tension grows, until the view pops into life (or occasionally doesn’t). It’s the moment I dread as I know everyone is staring at the circle, waiting. Seems like it makes 5 seconds feel like 50.

So here are my issues with this.

  • Positioning – The spinner is bang in the middle of the screen, you can’t avoid it. It grabs your attention and also the attention of the room. Everyone starts watching it whether they want to or not.
  • Information – The spinner doesn’t give any indication of the progress of the operation. I’m not sure how it can, given the nature of the underlying queries but it’s still a problem.
  • Inconsistency – Often the spin rate slows slightly just prior to completion. But sometimes it speeds up again. So I think it’s about to complete, then it carries on. I know it’s just an animated gif but it still seems to occur occasionally.
  • Errors – I’ve had occasions when the connection is lost with the server for whatever reason, but the circle keeps going. That’s very misleading.

I think everyone accepts that applications which perform loading operations will generally have some sort of indicator. But I think the spinning circle can be improved.

facebook_standard_loading_animation

Users blamed the system

facebook_custom_loading_animation

Users blamed the app

The psychology of waiting is certainly an interesting subject. This post refers to a study conducted by Facebook that seems to suggest the type of indicator used can affect how the user perceives the problem. Admittedly the post doesn’t provide an accurate source for this but I thought it interesting nonetheless.

.

Screen Shot 2015-01-09 at 22.57.05

There are many recommendations to making the waiting game feel less of a drag

There’s plenty of chat on how to make that waiting game feel less of a drag to users. This post from UXMovement.com has a number of suggestions such as

  • Use backwards moving ribbings
  • Increase number of pulsations
  • Accelerate progress, avoid pauses

Also take a look at this article by Chris Harrison. The associated video shows the theory in action and there’s a detailed study of the theories available.

.

I wonder how much thought Tableau have given the format and positioning of the spinning circle as well as the science of perceived performance as opposed to actual performance. I think the consensus is that a progress bar is the best option but I know that won’t happen as Tableau can’t easily know the duration of a query. However, there are plenty of recommendations out there that may be worth considering. Why not make use of every trick in the book to make users feel they’re getting a faster experience?

There has been some chat on the Tableau forums about this (thanks @johncmunoz). It seems such an insignificant component but why not add as much polish to the tool as possible? Anyway, I’m no expert on this so maybe someone who knows the subject can supply more info.

Until then I’ll just think of appropriate 90’s synth-pop as the circle spins.

Regards, Paul

2 minutes with… Dan Montgomery of Slalom Consulting

2 mins with title2

Hello everyone.

Really pleased to have Dan on this month’s show. Without further ado…

VN: Thrilled to speak to you, how are you doing?

259065_10100567987350728_5848874_oThank you very much! I’m doing well. It’s a sunny day in San Francisco, California. So after this interview and some additional Tableau presentation prep-work for Slalom and the San Francisco Bay Area Tableau User Group, I’ll probably go for a run.

VN: So who are you then and what do you do?

I am Dan Montgomery, aka @danrmonty, an Information Management & Analytics Consultant for Slalom Consulting, Tableau’s North American Partner of the Year for 2013. I’m part of Slalom’s group that works to help organizations leverage the power of data to provide insight and awareness to problems they are experiencing or don’t even know are problems yet. Any time I get to use Tableau on those projects, I get really excited because it is probably my favorite data analysis tool, but my work history comes from an SAP BW and Microsoft BI stack background, so I’m pretty proficient in those tools as well.

VN: How did you start using Tableau? And how are you using it now?

A few years ago, I don’t remember how long, my boss told me to look into this tool he had heard about called Tableau. I didn’t have any training or preparation, so it was a very slow pickup initially, but pretty soon I was showing off different ways to look at and ‘play’ with data that tools we currently had on our team couldn’t do.

When I joined Slalom in 2012 in Chicago, where I’m originally from, I was only one of a handful of people that used Tableau and started to evangelize its use in that office, while also connecting with our other office that were using Tableau more prominently.

These days, I am using Tableau in my client work and my personal work. I’ll create PoC’s using Tableau to show clients the potential of Tableau on their own data, as well as lead server architecture and dashboard development for clients. Tableau is a great partnership for Slalom; clients really appreciate the quick turnaround time from design to execution, access their information from multiple devices, and being able to quickly share findings with anyone else.

VN: What has the impact been on your business?

Slalom’s business model offers both technology and business consulting, one of the big reasons I wanted to work there. This means that we can start working with a client to help a specific need, but as time goes on and needs change, Slalom usually has a service that can support the changing environment. Tableau plays an integral role on both sides of that equation. Sometimes our IM&A group is brought in to do data analysis in Tableau, which results in the need for business process improvements or more organizational effectiveness, which are services we offer.

Other times, it happens in reverse: we can be doing program management work or infrastructure analysis and then Tableau can help take the results and present it fun and interactive way to our clients, which may lead to more IM&A work or selling Tableau software to the client. Tableau not only makes a great tool to have in our IM&A arsenal, but also becomes a way to compliment and transition the services we sell.

VN: How have you seen Tableau make an impact on businesses?

Tableau has a way of making people excited or engaged in data in ways that were never possible before. Hands off mangers can have content delivered to them in a number of flexible ways (email, mobile, web) and the dashboards can be designed to highlight a couple of key metrics or run the gambit of KPI’s. Hands on managers who used to have a series of analysts produce a series of reports and spreadsheets are now able to connect and drive the analysis themselves. Lastly, developers have a means to quickly respond to requests from said managers, while also share data sources they create with other developers to help manage a single source of truth.

We’re currently living in a data revolution and tools like Tableau are allowing people to stay up to speed.

VN: Who do you learn from in the Tableau community?

Oh my, so much. I really was living in a bubble for years when it came to Tableau development. Because I was one of the few people in my work circles that knew it, I was essentially a big fish in a small pond. Nelson Davis, an incredible Tableau developer and friend of mine, basically schooled me one day last fall, and from that moment on I’ve opened my eyes to the Tableau community and what people are able to do with the software.

There are too many to list, but these are my favorite developers to follow

  • Anya A’hern – Put simply, she makes Tableau art.
  • Kelly Martin – Never a lesson in simplicity that I can’t learn from Kelly.
  • Ramon Martinez – He takes incredibly complicated ideas and makes them digestible.
  • Mark Jackson – He is always pushing the boundaries of what you can do with Tableau and data visualization
  • Ben Jones, Matt Francis, Ryan Sleeper, Allen Walker, Paul Banoub, Jewel Loree,
  • Andy Kriebel – Continuing to put great content into the community and actively supporting others on twitter
  • John Mathis, Peter Gilks, Nelson Davis, Steven Carter – My Slalom partners in crime who regularly challenge and inspire me.
  • Karunaker Molugu – On the rise, just got his first VOTD and represents Chicago, my home town.

VN: In your opinion what should we be mindful of in the BI space going forward?

Respect for data and governance of data is the area I see as being a huge area that will be impactful in the near future. For years, data was ‘guarded’ simply by the fact that if it was in a database, only SQL developers knew how to access to it. Reporting tools like Tableau make data easier to access and interact with, and data storage platforms are making data available closer and closer to real time while also storing larger and larger datasets. ‘Turning everyone loose’ on data will not lead to more insight, and in fact may drive everyone further away from the truth.

Data isn’t just the numbers and text, it also includes data standards for how to represent information. This includes color schemes (what colors indicates good vs bad, progress vs decline, is this a gradient or is it stepped color), images (what are the accepted logos for different business units or vendors), fonts, and so many more areas that, as Tableau developers know, really become the measure of consistency and usability. Managing proper data definitions, hierarchies, calculated measures, etc. becomes more difficult and demands more attention, but the result is an empowered workforce that removes single points of failure and ensures consistent messages, regardless of the audience.

VN: Could you give me an interesting non-work fact about yourself?

1175648_10102743701092948_1357087646_nI never understood running when I lived in Chicago. Even though Chicago has an incredible running community, there’s only about 3-4 months out of the year where the weather isn’t trying to freeze you to death or melt you into a puddle. Now that I’m in California, where it’s between 50 and 65 F every day, I find I’ve really enjoyed getting into running. I’m actually running in a 191 mile relay the weekend of my birthday (May 2nd Slalom teams running to support organ donation. You can learn more and donate here if you would like. On top of that, I play competitive flag football to live a piece of my dream to play for the Chicago Bears.

VN: Thanks for your time Dan, see you soon.

Thank you for having me, Paul. I appreciate the support you’ve given me since becoming a part of the Tableau community and this was a great honor. Take care.

Awesome stuff. Tune in next time for more insight into the minds of the great and the good of data viz.

Cheers, Paul