|
Presented at the ETMA Conference 2022 in Nashville, TN - 10/05/2022
This is an update to a post originally made in February 2020.
11 obstacles you will encounter using traditional RPA (Robotic Process Automation) bot platforms for process automation on websites RPA on websites Traditional RPA bot platforms are powerful tools, however, if you are attempting to use them for automation on websites you will need to ensure you have solutions for the following 11 obstacles. 1. Logins The processes you want to automate may be on the Deep Web (i.e. behind login credentials). To automate these processes you’ll need to ensure your solution can securely store and/or access your credentials. This means both vetting the security posture of your bot automation platform and supporting integrations to 3rd party credentials vaults like Basis Theory, 1 Password, Amazon Secrets, and others. In addition, credentials change and may not always be updated in your vault or bot automation platform, which means your bots must be smart enough to recognize legitimate invalid credentials and not only surface visibility to your operations personnel for action, but also ensure that your bots don’t inadvertently lock logins by reattempting invalid credentials. 2. MFA - Multi-Factor Authentication Websites may require MFA when logging in from a new or unrecognized web browser. In addition, this step can be triggered at the discretion of a website at any time. You’ll need a solution to securely minimize, keep up with and resolve various types of MFA including OTP (One Time Passcode) to email, text, or authenticator apps. 3. CAPTCHA Websites may require you to solve CAPTCHA before proceeding with your automation. You’ll need a solution to solve, avoid and keep up with various types of CAPTCHA, which are constantly evolving to be ever more intricate and even invisible to the human eye. 4. Bot Deterrents CDNs - Content Delivery Networks More and more Websites deploy CDN’s (e.g. Cloudflare, Akamai, etc.) which may block bots intentionally to minimize bot traffic or unintentionally through their efforts to prevent DDOS (Distributed Denial of Service) attacks and improve general website performance. These deterrents are triggered by accessing a website in a way that distinguishes a bot from a human such as too fast, too frequently, concurrently, from non-trusted IP addresses, while using headless browsers, and various other reasons. You’ll need a solution to avoid and adjust to constantly evolving bot deterrents. 5. Website Changes When you automate on websites you’ll need to be prepared to manage and adapt to change. Page layouts are rarely static due to alerts, ads, and popups. Website DOMs (Document Object Models) are constantly evolving with element tags regularly changed or inconsistently named. Items on webpages may change their position rendering XY coordinates as a sole method for locating targets insufficient. To automate stably on websites, without overwhelming your resources, you’ll need a solution that can automatically solve for these changes and only surface the changes to operations that require human judgment. 6. Infrastructure As your automation efforts gain traction, eventually you’ll need to scale, which means you’ll need to move your bots to the cloud. This will require DevOps time and expertise to deploy, manage and scale your bots in the cloud. You will also need to adjust your bots to work with the various nuanced configurations, security, and monitoring differences you’ll experience running automations in the cloud. 7. Scheduling, Intermittent Downtime, Time-Outs, and Latency You’ll need a flexible solution to schedule the frequency of bot attempts and reattempts (for failed) unattended automations. Reattempts are particularly important with automation on websites, as websites may experience temporary downtime or slow page loading, which can cause bots to fail. Reattempt scheduling needs to be built into your bots and configurable to minimize manual intervention and maximize the efficiency of your automations. You’ll need a solution that can handle this detail and flexibility for scheduling. 8. New Front End Frameworks and Web Widgets As the web evolves so do the frontend frameworks and web components. Your bots should be able to run in any framework such as AngularJS and ReactJS and compatible with a broad array of web components such as modals, hidden fields, shadow DOM, and sophisticated navigation and search components. In addition, your bots should evolve to account for modern web development practices. With traditional RPA, this may mean building new bots or rebuilding your existing bots. 9. A/B Testing Websites can have multiple versions depending on region, customer type, and A/B testing. To adequately support this common scenario you will have to build and support multiple versions of your bot and ensure your automation tool is intelligent enough to timely identify which version of the Bot should be used for each Job. 10. Rogue Scripts, Insecure Connections, and Mixed Content Some websites include rogue scripts or libraries, which will trigger timeouts, and others insecure connections or content, which result in modern web browsers rejecting all or part of the page. You’ll need a solution to manage these obstacles. 11. Compliance The processes you want to automate may require or access sensitive information like PII (personally identifiable information) or financial data (e.g. credit card, banking information, etc.). This means you must validate that your solution is both secure and within compliance for your automation use case. (e.g. PCI DSS, GDPR, SOC2 Type II, etc.). Why Weeldi While traditional RPA platforms can be powerful tools, they often require costly, intricate configuration and even custom development to deliver an enterprise-ready solution for stable, scalable process automation on websites. Alternatively, considering purpose-built solutions for this use case could save you time and money. As an example, companies like Weeldi provide solutions to these 11 obstacles and many others, out-of-the-box, with no costly configuration and with no coding required.
San Francisco, CA — Weeldi, the easiest way to automate any task you do on a website w/ no coding required, announced today that it has successfully received PCI DSS Level 1 Certification, making it the first no-code, fully cloud-based website automation solution to receive this designation.
As companies and application providers look to automate tedious manual payment with thousands of vendors worldwide who lack the APIs for automated electronic payments, Weeldi opens up endless PCI Compliant payment automation possibilities by enabling non-developers to build stable, scalable payment automations in just minutes. From day one, Weeldi has helped customers automate on websites where traditional RPA tools, web drivers, and homegrown solutions struggle to deliver a stable and scalable automation experience, including in areas like behind logins, past OTP (One-time Passwords), through bot deterrents, on dynamic websites, around CAPTCHAs, with shadow DOM, between website downtime and a growing list of other hurdles. “We are very excited and proud to receive our PCI Level 1 Certification. This is the highest level of assurance a service provider can receive and it unlocks huge opportunities for us in the payment and procurement automation space,” says Moe Arnaiz, Co-Founder and CEO of Weeldi. “There are thousands of vendors worldwide like telcos, utilities, technology providers, shipping providers, and several others that don’t have (or make accessible) the APIs or other electronic means to support the payment and procurement automation their end customers want. With a few extra minutes invested, these customers can now use our PCI Compliant, website automation solution to automate these processes securely on the same customer-facing websites they’re using to do this work manually today. PCI DSS (Payment Card Industry Data Security Standard) is a set of security standards created by the credit card industry to protect payment systems from data breaches. Service providers that process more than 300,000 transactions annually are required to meet the higher PCI Level 1 requirements. Weeldi’s PCI Level 1 Service Provider Certification comes on the heels of achieving its SOC 2 Type II attestation in August 2022. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on websites, at scale, through its web service API or user interface — with no coding required. SAN FRANCISCO, CA — August 29, 2022: Weeldi, the easiest way to automate any task you do on a website w/ no coding required, announced that it has successfully completed the System and Organization Controls (SOC) 2 Type 2 audit in accordance with attestation standards established by the American Institute of Certified Public Accountants (AICPA). Conducted by risk3sixty, a leading professional services firm, the attestation affirms that Weeldi’s information security practices, policies, procedures, and operations meet the SOC 2 Type 2 standards for security, availability, and confidentiality. As companies use outside vendors to perform activities that are core to their business operations and strategy, there is a need for more trust and transparency into cloud service providers’ operations, processes, and results. Weeldi’s SOC 2 Type 2 report verifies the existence of internal controls which have been designed and implemented to meet the requirements for the security principles set forth in the Trust Services Principles and Criteria for Security. It provides a thorough review of how Weeldi’s internal controls affect the security, availability, and processing integrity of the systems it uses to process customers’ data, and the confidentiality and privacy of the information processed by these systems. This independent validation of security controls is crucial for customers. “Obtaining our SOC 2 Type 2 certification reinforces Weeldi’s ongoing commitment to the security, availability, and processing integrity of Weeldi’s no-code automation platform,” says Mathieu Guilmineau, Co-Founder & CTO of Weeldi. “Our customers can feel confident that we are taking security seriously at Weeldi.” In addition to a SOC 2 Type 2 attestation, Weeldi continues to make enhancements to its infrastructure by adding additional layers of redundancy and increasing monitoring coverage of its platform, while ensuring it remains the easiest and most stable way to automate any process customers do on a website. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on websites, at scale, through its web service API or user interface — with no coding required.  Quickly add low code instructions to your Templates in Weeldi to account for items that do not show up in your recording or are not easily recorded. (e.g. OTP resolution, CAPTCHA resolution, looping and extracting data from an entire table) Use Weeldi to automate past modern (multi-factor authentication) MFA by Authenticator Apps Use "Edit Events Timeline and Input Value Tables" to quickly edit your Weeldi automation w/ No Coding Required.
How Weeldi helped TruNorth automate and optimize data collection, enjoy immediate ROI, and even expand its service offerings
There are essentially two wins in business: increasing profits and decreasing expenses. The former is contingent on myriad factors: competition, economic trends, global politics, ad infinitum. The latter is typically contingent on a business itself – its processes, platforms, and people. The latter is also why TruNorth exists, providing expense management solutions and services to a growing roster of clients across most every major vertical market. Or, as they put it, bringing sanity and savings to clients with tools and information for easy governance over technology inventory and spend. However, expense management, in itself, can be expensive. Especially now. And as external costs and internal workloads were rising, TruNorth did not want to pass any of those burdens on to their clients. TruNorth turned to Weeldi to automate critical processes, and found that lowering expenses for themselves and their client base was not only easy–it was automatic. But that was just the start… The problem: 1,000 tasks a day… for starters Business technology expenses are typically endless, and vary wildly between businesses. For TruNorth to meet client needs, it had to execute literally thousands of different tasks daily, collecting, centralizing, and validating data locked inside PDFs, behind logins, across multiple systems and countless accounts. And it had to execute across specific business needs and processes: auditing, contract management, tech inventory, telecom management, and more. It found that its most talented team members were spending endless hours manually downloading expense data as the portals and apps where that data lived were rarely, if ever, accessible via APIs. TruNorth was in a rut of monotonous and mounting task work, which left less time to improve service offerings, or expand its lines-of-business, or focus on its own strategic growth. No code automation to solve today’s challenges–and tomorrow’s TruNorth turned to Weeldi for a solution, and Weeldi delivered. Better, Weeldi empowered. Weeldi’s no code, cloud-based application enabled TruNorth to build automations to capture expense data with no manual intervention. Unlike traditional RPA tools and apps like Selenium–which are limited toolsets–the Weeldi engine works directly through web drivers, which enables it to navigate through non-responsive sites, dynamic sites, and common automation deterrents like OTP, captchas and web blocking technologies like Cloudflare, Akamai and others. The data extracted by Weeldi is automatically deposited into TruNorth’s SFTP (secure file transfer protocol) where it’s cleaned-up, codified, and delivered into TruNorth’s expense management platform (Sparrow) where clients quickly and easily access their info. Security, flexibility, and support As Weeldi is a no-code solution, TruNorth is also empowered to quickly build new automations in just minutes, ensuring it’s equipped to handle any challenges future clients–and their portals, and their websites, and any obstacle between TruNorth and the data it needs–may present. Regarding security, credentials are securely stored outside of Weeldi in 1Password, giving TruNorth full control of automation credentials–a critical feature for a business working with sensitive financial data. Weeldi only accesses credentials as automations run, and does not store them. Weeldi’s dual focus on flexibility (no-code building) and security (no password storing) gives TruNorth and its clients a rare combination of speed and safety. TruNorth enjoys further peace-of-mind by having access to Weeldi’s support team to help solve unusual cases, add clarity to processes or outcomes, or simply bounce ideas off of. Weeldi ensures its clients have all the tools and training required to self-serve, but is also there to help when needed. From expenses to expansion Weeldi is currently automating data retrieval tasks across nearly 50 TruNorth vendors, and has delivered near-immediate ROI for the company. “But wait: there’s more.” Once implemented (in a matter of a couple of days), Weeldi also helped TruNorth uncover even more critical data, and deliver it on a fixed schedule (rather than TruNorth’s old way of doing things, which was based on client requests and other ad hoc processes). TruNorth now has more, and more actionable, data and more predictable workloads and workflows. But perhaps the most important deliverable TruNorth received was opportunity. Expense management is essentially the tip of the iceberg in the financial services space. The positive client relationships TruNorth had in place grew only stronger with the addition of Weeldi, as expense management processes were improved across the board. With strong client relationships, proven project expertise, and now–thanks to Weeldi–much, much more time available, it was only natural for TruNorth to expand its suite of services with a new outsourced bill payment offering, and thereby drive and deliver more value throughout its client base. “None of us wanted to spend our day downloading PDF invoices—now Weeldi’s robots do that for us while we sleep,” said Rick Lindner, Chief Strategy Officer, TruNorth. “But the value of Weeldi goes beyond that: it literally enables us to grow our business faster, more profitably and in new markets. Without Weeldi, we literally would not have been able to offer our new Bill Pay service.” Tapped to help TruNorth simply automate tasks related to expenses, Weeldi was able to extend its value and help TruNorth expand its business, delivering the two wins: lower costs, higher profits. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required. Here we provide an example of adding a Long Distance plan on a Telco Provider vendor portal.
10 obstacles you need to consider when automating past MFA. Automating repetitive tasks on the web can save your business time and money while enabling your human capital to focus on more strategic tasks. However, as websites beef up their security posture by implementing Multi-Factor Authentication (MFA) this security feature can be a hindrance to your automation objectives. If you’re automating behind logins with MFA, you’ll need to ensure you have solutions for the following 10 obstacles. 1. Configuration You’ll need a place for your One-Time Passcodes (OTP) to go, so you’ll have to set up an email inbox to capture OTP email forwards. Further, if you’ll be forwarding these emails to an outside domain you’ll have to make sure your email system is configured to accommodate forwarding from one outside domain to another outside domain. 2. Spam Filters If you’re not receiving OTP codes you’ll need to check your spam filter. Email spam filters may automatically block emails from the websites sending you OTP codes. If this is the case you’ll have to adjust your email system spam filter accordingly. 3. Non-Email MFA (SMS or Authenticator Apps) To further bolster security, more-and-more websites are moving away from email and require SMS or Authenticator Apps (e.g. Auth, Duo, Google Authenticator, etc) for MFA. You’ll need a solution that can capture and resolve these methods of MFA in an automated fashion. 4. Identifying OTP Codes For security purposes, OTP codes are generally sent without referencing a specific login. This can make it difficult to correlate OTP codes with their associated logins while you’re running concurrent automations. You’ll need a solution to ensure you can properly correlate OTP codes with their respective logins. 5. OTP Delays and Failures Due to various system issues, OTP codes don’t always arrive in the same order as you requested them, or in some instances, they don't at all. This means you’ll not only need a solution to properly correlate OTP codes with their respective logins but also automatically reattempt the OTP process if the correct passcode cannot be identified or never arrived. 6. Inbox Noise Often the same email address that receives OTP codes will also receive various other automated emails such as confirmations, notices and marketing messages, and/or other manual email communication. You’ll need a solution to sift through this email noise and hone in on the correct OTP codes. 7. Email Purging Data security is critical and since the same email addresses that receive OTP codes will likely receive various other emails, which may contain sensitive data and high volumes of data, it is critical that you are permanently purging your OTP inbox frequently to avoid accidental data leaks or a full email inbox. 8. Scaling Automated OTP Resolution As your automation volume grows, you will likely run into email throttling by your email provider and increased difficulty in associating OTPs to the proper login. You’ll need a solution to ensure high automation volume doesn’t break your automated OTP resolution process. 9. OTP Code Extraction from Emails Extracting the correct code from an email also presents its own challenges such as other sequences of numbers throughout the email body, layered multipart email format, base64 content-encoding, or even embedding the code in an image. You’ll need a solution to ensure you can extract the OTP code accurately in these scenarios. 10. Minimizing OTPs Similar to your experience accessing websites manually, OTPs generally don’t present themselves on every login unless they have been configured to do so. However, if your login looks suspicious: like it comes from a bot, or like it comes from an unrecognized browser, you’re likely to be presented with OTP on each login. This can slow down automations and result in more failures. You’ll need a solution to minimize the presence of OTP once you’ve passed it once. Why Weeldi While Multi-Factor Authentication (MFA) is a powerful tool to bolster website security it can become a hindrance to scaling your automation objectives. Attempting to solve MFA challenges with custom code is a perpetual, resource-intensive effort that can distract from your focus on automating. Companies like Weeldi provide an out-of-the-box solution to these 10 MFA obstacles and many others with no costly configuration or coding required. In addition, you pay for the results, not for integrator hours, development hours, and/or per bot hours that may not result in your long-term automation success. Learn how to extract text from a web page with a couple clicks and No Coding required. 🎥 Extracted text made available via the Weeldi UI, a CSV deposited into your FTP or the Weeldi API. ETMA Thought Leadership features Weeldi 10 Obstacles to Automate Telecom & Technology Vendor Portals12/22/2021
Weeldi Co-Founders, Moe Arnaiz (CEO) and Mathieu Guilmineau (CTO) join Joe Basili at ETMA to discuss automation on the web. Use the "Add-Context" feature when recording a Weeldi automation on a very busy web page with multiple data points, named similarly, but refreshing at different times. 🎥 Weeldi Automation Analyzer helps our prospects and customers clarify where Weeldi can automate and understand key elements of their automation target website. ✔️ SAN FRANCISCO, CA — September 23, 2021: Weeldi, the easiest way to automate any task you do on the web w/ no coding required, announced that it has successfully completed the Service Organization Control (SOC) 2 audit. Conducted by risk3sixty, a leading professional services firm, the audit affirms that Weeldi’s information security practices, policies, procedures, and operations meet the SOC 2 standards for security, availability, and confidentiality. As companies use outside vendors to perform activities that are core to their business operations and strategy, there is a need for more trust and transparency into cloud service providers’ operations, processes, and results. Weeldi’s SOC 2 report verifies the existence of internal controls which have been designed and implemented to meet the requirements for the security principles set forth in the Trust Services Principles and Criteria for Security. It provides a thorough review of how Weeldi’s internal controls affect the security, availability, and processing integrity of the systems it uses to process customer’s data, and the confidentiality and privacy of the information processed by these systems. This independent validation of security controls is crucial for customers. “Obtaining our SOC 2 certification reinforces Weeldi’s ongoing commitment to the security, availability, and processing integrity of the Weeldi’s no-code automation platform,” says Mathieu Guilmineau, Co-Founder & CTO of Weeldi. “Our customers can feel confident that we are taking security seriously at Weeldi.” In addition to SOC 2 compliance, Weeldi continues to make enhancements to its infrastructure by adding in additional layers of redundancy and increasing monitoring coverage of its platform.
How Weeldi's no-code solution helped a fast-growing company bridge the gap between two systems… by creating an API where one didn't exist.
Wondering if solar energy use is actually soaring? Look no further than the massive success of Powur, a residential solar company that has grown 5890% in the last year. That’s five-thousand-eight-hundred-and-ninety percent, which put it at #61 on the 2021 Inc. 500 list. Massive growth often means massive stress on current processes and infrastructure. Powur knew the only way to keep growing and concurrently reduce that stress was to further automation — and do so intelligently. This meant optimizing existing workflows and data-streams and fueling automation across business systems and applications. A problem with proposals Powur’s first area of focus was sales proposals, the signed quotes that essentially affirm a company is growing, the documents containing all the actionable info connected to a new opportunity, a new customer, and new revenue streams. Powur was using its own proprietary system to track and manage sales, customer relationships, finance and operations, but using a third-party cloud-based SaaS tool to create, track, and manage sales proposals. The challenge was that the tool lacked the API functionality for Powur to automate critical, downstream, workflows into its system — the processes that drives installs, billing, and commissions. Without an API, the only solution to collect the data buried in the SaaS tool’s user interface and the web-based PDFs and emails it generated was endless — and costly — hours of manual labor. And when a company is growing at nearly 6000% a year, manual labor only gets in the way and slows down the pace. Enter Weeldi. From a manual mess to an automated API Headquartered in the San Francisco Bay Area, Weeldi provides the easiest way to automate any process on the web with no coding required. A common use case for Weeldi is filling gaps — where vendor portals and/or SaaS applications lack API functionality — and enabling customers to automate tasks directly through a vendor portal or SaaS application user interface. Common tasks for Weeldi include extracting data, executing transactions, and paying bills. In short, Weeldi turns an unwieldy data collection process into one that is wieldy/Weeldi. The problem Powur was facing and the solution Weeldi offered were an exact fit. Using Weeldi, Powur was able to:
The automation is kicked off by an email forwarded from the solar proposal tool to Weeldi. Weeldi then goes behind the tool’s login and scours multiple web pages and web-accessible PDFs to pull down and structure the data Powur needs for their downstream process. An even brighter future lies ahead With Weeldi automating data collection and integration between its sales proposal tool and its backend system, Powur has very little slowing it down as it continues its winning streak. “Weeldi has reduced our data entry labor costs by 80%, and reduced our contract processing time by 90%,” said Dean Rosenberg, Chief Operating Officer at Powur. “We’re already planning on using Weeldi in other cases where we require data or need to execute transactions from applications or websites without available or reliable APIs.” In addition to the clear benefits delivered by automating data extraction and delivery with Weeldi, Powur also benefitted by standardizing current workflows simply by going through the process of automating with Weeldi. The project was a success on an implementation level based on the results it is producing for Powur, but also on an educational level by creating a thought model to engineer future processes that can be automated. In one year, burdened with time-consuming manual labor attached to one of its most critical and revenue-centered workflows, Powur was still able to grow 5890%. Going forward with Weeldi…? The sun’s the limit. About Powur PBC: Founded in 2015, California-based Powur is a Public Benefit Corporation that is on a mission to accelerate the adoption of solar energy worldwide. As the first 100% virtual, residential solar company in the cloud, Powur operates an innovative platform model, enabling it to scale quickly and meet the demands of an ever-expanding solar landscape. The company’s model ensures that Powur can provide affordable and customized solar solutions to homeowners while empowering its network of thousands of independent solar consultants and regionally specialized installation partners. Powur is the fastest growing residential solar company in the US, currently operating in 15 states and completing dozens of residential solar projects, and adding to Powur extended family every working day. For more information on the company, visit https://powur.com/.
A common misconception is automation on the web is unstable, high maintenance, and hard to scale because websites simply change too often — resulting in a constant cat-and-mouse game.
However, after looking at over 1M automations, over multiple months, on Weeldi the numbers tell a different story. 45% of automations fail because the customer input data is wrong. These are issues like invalid log-in credentials, invalid account numbers, incorrect scheduling, incorrect payment information, and attempting to capture data that doesn't exist. Weeldi’s API and User Interface allows users to update this input data on demand and classifies errors by customizable error type with supporting screenshots, so users can quickly (and automatically) identify and fix automations failing due to incorrect input data. Ultimately, if you want automation to succeed you need your input data to be clean and up-to-date. 43% of automations fail because of multiple website versions. This is most commonly believed by users to be a change with a website, however it is instead a new version of the same website, which they have yet to come across while automating. As you scale the Job volume behind each automation you see different scenarios you didn’t see before. This is commonly a result of regional differences, AB testing, or different account types on the same website. Weeldi’s automation engine supports multiple flows per website allowing customers to record multiple automations per website. Then Weeldi uses AI to figure out which automation works per Job and memorizes this for future runs, increasing automation success and removing painful guesswork for users. Simply put, volume strengthens automation. 7% of initial automations fail because of website performance issues. This means the website is down, loading too slowly, not returning a query. Generally, these are temporary issues and occur during website updates or more often on websites pulling data from several legacy systems, as at any given moment one of those systems may be down. (e.g. vendor websites like utilities, telcos, suppliers, etc.) Weeldi enables reattempt scheduling by automation, which means if a website is down, slow, or not returning an expected result, Weeldi will reattempt within a configurable timeframe until the website can complete the automation successfully. 4% of automations (if using RPA) will fail because of changing elements. Websites are built using HTML tags. A common example, a pay bill button may be implemented as element type BUTTON, but upon a new website update, it may have been changed to DIV, SPAN, or INPUT. These are tricky because while nothing may change to the human eye, these tag changes will stop traditional automation tools like RPA dead in their tracks. Since Weeldi automatically solves for these changes (as described below) instead of tracking failures, we tracked how often our automation engine had to solve for these changes. Weeldi’s proprietary approach to automating on the web takes into account a more complete description of the webpage (not just HTML tags) and uses AI to add an abstraction layer above the webpage, allowing the Weeldi automation engine to view web pages like a human. This means if an HTML tag changes, an element moves, or an element is named differently, chances are very high Weeldi will navigate through it ensuring the automation completes successfully. .67% of automations fail because of wholesale website UI changes. Wholesale website UI changes are often assumed to be the primary reason for web automations breaking, however wholesale website UI changes are by far the least common reason why Weeldi automations break. The reality is wholesale UI changes take a lot of time and endanger the user experience, which means they don't happen frequently. Weeldi’s automation engine is equipped with tools that make it clear when automations are failing and why they are failing, including supporting screenshots and error details. This makes it immediately obvious when a complete website UI change has taken place. The next step is adjusting to the new change and this is where Weeldi's No Code Recording Engine (a Chrome Extension that runs in your Google Chrome Browser) allows non-technical users to quickly record a new or updated web automation in just minutes. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required.
How an up and coming machine learning technology overcame every barrier to automatically collect subscription and service data from vendor web portals and SaaS applications
MISO3’s line of business requires getting data in hard-to-reach places An intelligent subscription and service tracking company serving the needs of enterprise-level companies, MISO3 gives its clients insight into its vendor subscriptions and usage. Think about it: vendors like Zoom, Microsoft 365, Salesforce, Amazon AWS, CenturyLink, Bloomberg, and the list goes on (and on and on) — there are few businesses left that are not reliant on licenses, SaaS, PaaS, and/or IaaS subscriptions to power their business. The data required to manage these vendors spans inventories, portals, and invoices, making visibility as well as the ability to make data-driven decisions almost impossible. For most clients, MISO3 was able to deliver on its promise rapidly and with few hurdles. And then… one day… MISO3 came up against a challenge, and a big one. A Fortune 100 client and one of the best-known financial and credit card companies in the world came to MISO3 with the challenge of understanding and optimizing subscription costs for their financial data and intelligence provider. MISO3 knew exactly what it needed from the provider to better understand the client’s spend and utilization of their subscriptions and services. However, with no inventory-targeted APIs available and data scattered throughout a clunky provider portal, they needed a predictable and scalable way to get control of this multi-million-dollar expenditure. They turned to Weeldi. Headquartered in the San Francisco Bay Area, Weeldi is the easiest way to automate any process on the web with no coding required. A common use case for Weeldi is filling gaps (where vendor portals and/or SaaS applications lack API functionality) by enabling customers to automate tasks directly through the vendor portal or SaaS application user interface, common tasks include extracting data, executing transactions, and paying bills. For MISO3, Weeldi’s web automation platform would be the cornerstone of meeting its client’s need to get its vendor data out of multiple locations quickly, accurately, and with as little friction as possible. MISO3 prides itself on being the first “easy-button” in vendor inventory discovery, extraction, and alignment. What once took businesses months of labor-intensive research and data cleaning, can now be done in only hours; minimizing user-prone errors and discovering hidden costs that often uncover waste and security vulnerabilities. (The alternatives were hours of manual labor every month or attempting to engineer a homegrown web automation solution with all the costs, delays, and opportunities for error that creates — rarely an efficient path to client satisfaction.) Data: it’s everywhere, but still so hard to find A little background on the problem faced by MISO3’s client (and, really, all of its clients, and almost every business everywhere). It’s all about “the other guys”: the SaaS companies, the IaaS companies, and the PaaS companies —and the mixed blessing they provide. In 2020, the overall spend on SaaS products was up by 50% compared to only two years before. (Data has yet to be collated, but we’d bet it’s going up again in 2021.) The average company uses 137 different SaaS apps, with each employee using at least 8 different ones. (Thanks to Blissfully for this data.) With subscriptions varying from person to person and from department to department across some or all of those 137 vendors, most companies are tilting at windmills when it comes to effectively managing subscriptions. They’re oversubscribed, leaving ex-employees licensed (sometimes, to do ill), and generally overbuying across the board. They do have access to data to see what’s what, of course. But — because they need to manually chase and capture it — the data is rarely in real-time… or complete… or accurate. It makes a difference in the same way Monday-morning quarterbacking makes a difference. This is where MISO3’s M3 platform powered by data extracted from Weeldi helps. Overcoming logins and CAPTCHAs and other roadblocks—automatically Weeldi was brought in by MISO3 to solve these problems and more. What Weeldi does that others can’t is to create stable automations on vendor portals and SaaS applications that neither Weeldi nor its clients control. Weeldi ’s platform allows customers to create automations on the web when APIs simply aren’t enough, or when APIs simply aren’t available (e.g., for billing data and subscription info — the exact kind of data MISO3 collects for its clients). Weeldi has a menu of available vendor portal automations and provides the flexibility of “built to order” automations that customers can create in minutes with no coding required. Automations can uncover and capture the data or execute transactions buried in vendor portals (multiple pages, behind a login, multi-factor authentication, CAPTCHAs, requiring scheduling or filtering, on the user interface, in emails, on unstable websites, etc.) or within SaaS applications. Weeldi’s flexible and reliable automations for the web were the ideal fit for a client like MISO3, a company with a business model reliant on retrieving information from vendor portals and SaaS applications, and were central in MISO3 sustainably supporting the business of one of the best-known credit card and financial companies in the world. A solution delivered in days; results that last a lifetime Within days of receiving specifications from MISO3, Weeldi delivered complex automations that extract — from behind logins (and tricky multi-factor authentication) — critical data, some requiring filters by date and account number, some from the user interface, and even some embedded in emails. Files are then automatically renamed, and folders are created in MISO3’s sFTP to feed their ETL (extract, transform, load) process back into the M3 platform. This was not a “one and done” for MISO3, either. The automations created by Weeldi give it the ability to quickly create connectors to any vendor portal or SaaS application. Additionally, Weeldi provides precision, actionable visibility to MISO3 (and all Weeldi clients) on how their data is flowing. Weeldi can say exactly where and why the data isn’t flowing and provide direction (along with screenshots) and tools to fix it. In the end, MISO3 had a solution that served its immediate need with its client, and a solution it could use repeatedly as it continues to build its business. “Before you can make data-driven decisions for your organization, you must first know what you have,” says David Baule, CEO of MISO3. “Gathering this data manually is both cumbersome and time-consuming. Weeldi gives us the power to quickly scale to any web-based vendor portal or SaaS application at a fraction of the time of doing this manually. The result, quick and frictionless ingestion of customers’ data so M3 Users can make informed decisions on all of their vendor commitments.” Weeldi transformed disparate, dynamic, and sometimes unstable Vendor Portals into stable, scalable APIs, providing MISO3 with the detailed data required to analyze and optimize their customers' subscription costs. All delivered with no coding required by MISO3. A solution delivered in less than two weeks that will last a lifetime. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required. Use "Input Values" to identify the key variables that drive your Weeldi automation. This enables one automation to power many scenarios. 🎥 Learn how to record one Weeldi Template that will extract as many files as you need — using the "Add-Lookup" feature. 🎥 This video takes you through setting up email notifications in Weeldi. Use the Weeldi Chrome extension to build and maintain web automations in near real-time.
EXECUTIVE SUMMARY:
Motus is the definitive leader in solutions for businesses with mobile-enabled workforces helping over 2,300 companies and 300,000 mobile workers simplify reimbursement and management of mileage, mobile devices and remote work. After a recent acquisition, Motus faced the challenge of automating its acquisition’s data collection processes from over 20 vendor websites, spread across 13 countries and in 10 languages. Motus had a simple objective: quickly create automated processes to identify, capture, and consolidate the critical business data it needed from anywhere on the web it was living. So Motus explored every opportunity to automate data collection on the web. After a pilot with a well known RPA (robotic process automation) platform, and a failed experiment with an open source tool, Motus chose Weeldi for a pilot project. When Weeldi delivered five stable automations within its ten-day risk free pilot, Motus chose Weeldi as a partner. The company currently enjoys more than 90 automations in 10 different languages, collecting endless amounts of critical usage and spend data for their customers. The problem: complex automation of manual processes on the web to better handle scale The nature of the mobile workforce is often chaotic. Different devices used by different staff; working in different places and timezones; different needs for tech, transportation, and just about everything. As the company that simplifies management for all that (and more) for over 2,300 customers, Motus has to think and act at scale. In this case, it meant being able to consistently and completely collect critical data for its clients from multiple vendor websites regardless of the hurdles at each step (CAPTCHAs, Two-Factor Authentication, instability, different website frameworks, etc.). Prior to acquisition, the acquired company’s processes could be handled with the right people, manually. But after being acquired and integrated into Motus, the need for scale increased exponentially, and manually combing through thousands of webpages a month was no longer an option: automated processes were needed to prepare for the increase in volume, while the manual labor was directed to higher value tasks. The search for a solution, part 1: start, sputter, stop After working with a well-known RPA (robotic process automation) company to create a proof-of-concept for a single automation, Motus turned to Selenium, an open-source browser automation tool (primarily used for app testing) to build it. After a development process that took the better part of two months, one automation was built. However, it proved to be a house of cards, unstable and unable to accomplish the single task it was developed to execute on the web. So Motus looked elsewhere and decided on a 10-day pilot with Weeldi, a San Francisco-Bay Area company that enables companies of any size or technical ability to stably automate processes on the web, at scale, with no coding required. The solution, part 2: automation at a record-setting pace, and we’re just getting started In the 10-day pilot, Weeldi delivered five full-featured automations to help Motus automate complex manual backend processes on vendor websites, and another 40 processes just days after the official start of the engagement. Weeldi was able to create stable automations on the web for Motus (without requiring any coding work from the client), solving challenges such as login, OTP resolution, task scheduling/completion, and more. Motus now enjoys more than 90 automations, in 13 countries, and 10 languages. Other “Weeldi wins” for Motus include:
With Weeldi, Motus enjoys the freedom to quickly create dependable automations for endless tasks across the enterprise and can better and more completely utilize the skill sets of its staff. Perhaps most importantly, because of Weeldi’s flexibility, Motus essentially has every automation on the web it will ever need at its fingertips. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required.
8 areas where you can leverage AI today for automation on the web.
Artificial Intelligence (AI) is a hot and highly marketed topic, but the application of AI for automation on the web is still in its infancy. This means those laying the right foundation today will be rewarded with the compounding value of less friction and more success with future automations. Here are 8 areas where you can begin leveraging AI for automation on the web. 1) Login Many automations on the web start by getting through a login page. AI can be leveraged to simplify the login step by looking up standard login components such as username, password, and “remember me” functionality, minimizing obstacles like MFA (Multi-Factor Authentication) and CAPTCHAs, and increasing the probability of automation success when there are login page changes. 2) Flow selection A growing number of websites have multiple versions based on region, customer type, M&A (Mergers & Acquisitions), and/or A/B testing. Using AI you can simplify the support of multiple website versions. This can be achieved by recording multiple automation flows, by website version, then using AI to select and navigate to the correct flow. This approach adds scale and stability to the automation by eliminating the need for a human to micro-manage variability of website versions. 3) Managing website issues Traditionally, website instability is a key culprit for breaking automations on the web. AI can help solve the most common instability issues like slow-loading webpages and webpage timeouts. It can do this by intelligently adjusting bot wait times by website, proceeding when critical components of a webpage are available (but noncritical components are not) and automatically reloading webpages with timeouts. 4) Formatting Bot automation flows are guided by data such as dates, account numbers, and SKUs. However, the way this data is presented on websites may vary from how you store it in your database, and even if you harmonize the two, eventual changes to the presentation of data on the web will break your automations. AI can help solve this problem by recognizing data patterns when websites make changes to the way they present data. Common examples are changes in date formats (i.e. 01/17/2021 vs. 17/01/2021), account numbers (i.e. 001234567 vs. 1234567), and SKUs (i.e. TESLA-S vs. Tesla Model S). 5) Nomenclature Change on the web is constant as websites evolve to meet their users' expectations. For example, you may download your utility bill each month by clicking on a link that states “Invoice”, but next month that same link may state “Bill”. These types of seemingly obvious changes to the human eye will break traditional bot automations. By using AI you can better recognize relevant nomenclature changes such as synonyms and increase your probability of success automating on the web. 6) Navigation Website changes go beyond nomenclature changes. Sometimes nomenclature stays the same, but the elements in the User Interface change. A common example is a website modernizing clickable links by transitioning them into buttons. This kind of change will break traditional bot automations. By storing the goal/intent of your automation rather than just the HTML element location and raw browser interactions you can leverage AI to successfully navigate website changes. 7) Pop-ups Websites use pop-ups to gain the attention of their users. Unfortunately, this can break brittle, bot automations. By using AI you can intelligently recognize pop-ups, the content within them and derive whether your automation should fail or close (or acknowledge) the pop-up and proceed with the automation. 8) Scheduling Some Bot automations run on a preset schedule based on an understanding the automation can be executed successfully at that time. But, what if your scheduled date is wrong. A common example is you command your bot automation to download a report on a specific date, but continually the report isn’t available until 2-3 days later. AI can help you solve this problem by learning the optimal schedule based on historical automation success and suggest the appropriate schedule adjustment. Why Weeldi? As companies attempt to automate predictably on the web they must consider the web's dynamic nature and look to pattern recognition technologies like AI to help manage this variability. Companies like Weeldi have started laying the groundwork by designing automations that focus on user intent, rather than exclusively on static navigation steps, and then using AI to successfully navigate to the users expected outcome. This approach proves to be a sturdy alternative for automation on the web where change is constant, broad, and unpredictable. Move past RPA and (other legacy tools) to automate successfully on telco web portals. |
WeeldiTransform a website into an API in seconds w/ no coding required. 
Archives
September 2023
Categories |
|
|
|
© 2023 Weeldi LLC. All rights reserved. |
|