Building a Regret-free Foundation for your Data Factory

Published On: 2021-09-19By:

Are you new to Azure Data Factory and wondering what you don’t know you don’t know? The learning curve with new technologies can sometimes lead to some major refactoring down the line once we realize our mistakes. Join Meagan Longoria and Kerry Tyler to learn how to set up your data factory for success. They will start by discussing naming conventions, parameterization, Key Vault usage, and deployment with Azure DevOps. Then they’ll share their recommendations on pipeline hierarchies, activity dependencies, error handling, and monitoring. Watch this webinar to help your organization avoid Data Factory regrets!

The post Building a Regret-free Foundation for your Data Factory appeared first on DCAC Videos.

Contact the Author | Contact DCAC

Slides and Video from Building a Regret-free Foundation for your Data Factory Now Available

Published On: 2021-09-16By:

Last week, Kerry and I delivered a webinar with tips on how to set up your Data Factory. We discussed version control, deployment, naming conventions, parameterization, documentation, and more.

Here’s our agenda from the presentation.

Slide showing top regrets of data factory users: Poor resource organization in Azure
Lack of naming conventions
Inappropriate use of version control
Tedious, manual deployments
No/inconsistent key vault usage
Misunderstanding integration runtimes
Underutilizing parameterization
Lack of comments and documentation
No established pipeline design patterns
List of top regrets from Data Factory users that they wish they had understood from the beginning

If you missed the webinar, you can watch it online now. Just go to the DCAC website, fill in the required fields with your info, and the video will be shown.

If you’d like a copy of the slides, you can download the PDF here. There is a list of helpful links at the end that you may want to check out.

I hope you enjoyed our webinar. Leave me a comment if you have other experiences with ADF where a design or configuration choice you made in the beginning was difficult or tedious to fix later. Help other ADF developers avoid those mistakes.

Contact the Author | Contact DCAC

Dramatically Decreasing the Time to Push Software to the Edge

Published On: 2021-09-13By:

Last week I was introduced to Avassa. As the presentation started I had no idea what their solution did. By the end of the presentation, I wanted to know more about how it worked and my mind was racing with the deployment options that I see from it, both for our existing clients as well as future clients.

The Avassa solution is designed to assist you in pushing docker images to the edge devices that on running within your environment. The demonstration that Avassa delivered at Tech Field Day 24 really drove home just how powerful their solution was. In their demo (which was probably 90% of their presentation) they had a fictitious movie theater company that had 100 theaters all over the world. Each theater had a small docker environment that was used to run the backend software that was needed to run the movie theater. Using the Avassa software they were able to deploy new images to all 100 theaters within seconds with just a few mouse clicks. Upgrading those docker images to a new version was just as simple. You simply tell the solution that there’s a new image available on GitLab and the solution downloads the image and pushes the image to all the edge devices.

The beauty of this solution is that network connectivity to the edge site doesn’t need to be stable. In the event that the network is unstable, the Avassa solution downloads the image and deployed the image once it’s been downloaded. So if the edge isn’t on the internet for days, weeks, or months, the command to upgrade that edge node will be processed as soon as it is back online. When I heard this I immediately thought of use-cases in fields such as oil and gas where you have oil rigs in the middle of the ocean and they aren’t always able to be connected to the Internet due to weather or satellite uplink issues, or the shipping/cruise industry where you’ve got ships that aren’t in their home port for weeks or months and we want to be able to push software changes to the ships as soon as they get internet access back.

Even edge devices that are always connected such as studios that have spin classes, retail operations, etc. could take advantage of this by lowering the time needed to deploy software to all the retail locations. When retail locations need to do a software upgrade, they will either have someone from the head office handling the upgrade through a remote desktop solution or if they need physical hands on a keyboard they will have a local IT partner that can put hands on a keyboard. However, either of these solutions is cost-free. But being able to push docker images to all the stores either in a rolling fashion or at scale that cost goes to next to nothing.

The ability to be able to quickly and easily push changes to the edge devices has always been a problem with environments which large numbers of edge devices. As the number of edge devices increases the management headaches which are involved with upgrading software increase.

If you’re in an environment with edge devices, I highly recommend checking out the videos from Avassa’s presentation at Tech Field Day 24. During the session, we pointed out some things to the Avassa team that we’d love to see improved in their UI, and the Avassa team’s response was basically “yes we know, it’s already in planning for a later release” which tells me that they are going in the right direction for sure, and I personally can’t wait to see what their team is able to do with their solution as it gets more mature.

Denny

Contact the Author | Contact DCAC

Data Community Summit 2021 Pre-cons

Published On: 2021-09-10By:

The team at DCAC is proud to say that we have members of our team presenting two different pre-cons at the inaugural Data Community Summit which is being held November 8th, 2021 – November 12th, 2021. Denny Cherry will be pressing a pre-con titled “Database Administration for the Non-Database Administrator” while Monica Rathbun will be presenting “SQL Server Performance Tuning and Optimization“.

Registering for one of these sessions is quick and easy. Simply register for the free conference, (you’ll probably need to select the Create Account option). After you fill out all the demographic questions, select the conference pass, which will get you into the free event which is being presented from November 10th-November 12th.

You select the free conference, select Monica’s session on Monday, and/or Denny’s session on Tuesday (there is an additional fee to attend these day-long sessions).

Then complete your registration.

If you have already registered for the free conference, then login to your existing account, and select the Add package button on the right.

From here you can select the precon sessions that you want to add to your registration and complete your order.

The cost for the pre-con sessions is $200 USD plus any VAT that’s due based on where you live and where RedGate has operations (as RedGate is hosting the event).

Don’t forget to register and we’ll see you on the 8th or the 9th.

If you have questions about the event, check out the event’s FAQ.

Denny

Contact the Author | Contact DCAC
1 2 3 495

Video

Globally Recognized Expertise

As Microsoft MVP’s and Partners as well as VMware experts, we are summoned by companies all over the world to fine-tune and problem-solve the most difficult architecture, infrastructure and network challenges.

And sometimes we’re asked to share what we did, at events like Microsoft’s PASS Summit 2015.

Awards & Certifications

Microsoft Partner   Denny Cherry & Associates Consulting LLC BBB Business Review    Microsoft MVP    Microsoft Certified Master VMWare vExpert
INC 5000 Award for 2020    American Business Award 2021 Gold    American Business Award 2021 Bronze    FT Americas’ Fastest Growing Companies 2021   INC 5000 Award for 2020    American Business Awards People's Choice    American Business Awards Gold Award    American Business Awards Silver Award
FT Americas’ Fastest Growing Companies 2020       Best Full-Service Cloud Technology Consulting Company       Insights Sccess Award    Technology Headlines Award
Share via
Copy link