|
SAN FRANCISCO, CA — September 23, 2021: Weeldi, the easiest way to automate any task you do on the web w/ no coding required, announced that it has successfully completed the Service Organization Control (SOC) 2 audit. Conducted by risk3sixty, a leading professional services firm, the audit affirms that Weeldi’s information security practices, policies, procedures, and operations meet the SOC 2 standards for security, availability, and confidentiality. As companies use outside vendors to perform activities that are core to their business operations and strategy, there is a need for more trust and transparency into cloud service providers’ operations, processes, and results. Weeldi’s SOC 2 report verifies the existence of internal controls which have been designed and implemented to meet the requirements for the security principles set forth in the Trust Services Principles and Criteria for Security. It provides a thorough review of how Weeldi’s internal controls affect the security, availability, and processing integrity of the systems it uses to process customer’s data, and the confidentiality and privacy of the information processed by these systems. This independent validation of security controls is crucial for customers. “Obtaining our SOC 2 certification reinforces Weeldi’s ongoing commitment to the security, availability, and processing integrity of the Weeldi’s no-code automation platform,” says Mathieu Guilmineau, Co-Founder & CTO of Weeldi. “Our customers can feel confident that we are taking security seriously at Weeldi.” In addition to SOC 2 compliance, Weeldi continues to make enhancements to its infrastructure by adding in additional layers of redundancy and increasing monitoring coverage of its platform.
How Weeldi's no-code solution helped a fast-growing company bridge the gap between two systems… by creating an API where one didn't exist.
Wondering if solar energy use is actually soaring? Look no further than the massive success of Powur, a residential solar company that has grown 5890% in the last year. That’s five-thousand-eight-hundred-and-ninety percent, which put it at #61 on the 2021 Inc. 500 list. Massive growth often means massive stress on current processes and infrastructure. Powur knew the only way to keep growing and concurrently reduce that stress was to further automation — and do so intelligently. This meant optimizing existing workflows and data-streams and fueling automation across business systems and applications. A problem with proposals Powur’s first area of focus was sales proposals, the signed quotes that essentially affirm a company is growing, the documents containing all the actionable info connected to a new opportunity, a new customer, and new revenue streams. Powur was using its own proprietary system to track and manage sales, customer relationships, finance and operations, but using a third-party cloud-based SaaS tool to create, track, and manage sales proposals. The challenge was that the tool lacked the API functionality for Powur to automate critical, downstream, workflows into its system — the processes that drives installs, billing, and commissions. Without an API, the only solution to collect the data buried in the SaaS tool’s user interface and the web-based PDFs and emails it generated was endless — and costly — hours of manual labor. And when a company is growing at nearly 6000% a year, manual labor only gets in the way and slows down the pace. Enter Weeldi. From a manual mess to an automated API Headquartered in the San Francisco Bay Area, Weeldi provides the easiest way to automate any process on the web with no coding required. A common use case for Weeldi is filling gaps — where vendor portals and/or SaaS applications lack API functionality — and enabling customers to automate tasks directly through a vendor portal or SaaS application user interface. Common tasks for Weeldi include extracting data, executing transactions, and paying bills. In short, Weeldi turns an unwieldy data collection process into one that is wieldy/Weeldi. The problem Powur was facing and the solution Weeldi offered were an exact fit. Using Weeldi, Powur was able to:
The automation is kicked off by an email forwarded from the solar proposal tool to Weeldi. Weeldi then goes behind the tool’s login and scours multiple web pages and web-accessible PDFs to pull down and structure the data Powur needs for their downstream process. An even brighter future lies ahead With Weeldi automating data collection and integration between its sales proposal tool and its backend system, Powur has very little slowing it down as it continues its winning streak. “Weeldi has reduced our data entry labor costs by 80%, and reduced our contract processing time by 90%,” said Dean Rosenberg, Chief Operating Officer at Powur. “We’re already planning on using Weeldi in other cases where we require data or need to execute transactions from applications or websites without available or reliable APIs.” In addition to the clear benefits delivered by automating data extraction and delivery with Weeldi, Powur also benefitted by standardizing current workflows simply by going through the process of automating with Weeldi. The project was a success on an implementation level based on the results it is producing for Powur, but also on an educational level by creating a thought model to engineer future processes that can be automated. In one year, burdened with time-consuming manual labor attached to one of its most critical and revenue-centered workflows, Powur was still able to grow 5890%. Going forward with Weeldi…? The sun’s the limit. About Powur PBC: Founded in 2015, California-based Powur is a Public Benefit Corporation that is on a mission to accelerate the adoption of solar energy worldwide. As the first 100% virtual, residential solar company in the cloud, Powur operates an innovative platform model, enabling it to scale quickly and meet the demands of an ever-expanding solar landscape. The company’s model ensures that Powur can provide affordable and customized solar solutions to homeowners while empowering its network of thousands of independent solar consultants and regionally specialized installation partners. Powur is the fastest growing residential solar company in the US, currently operating in 15 states and completing dozens of residential solar projects, and adding to Powur extended family every working day. For more information on the company, visit https://powur.com/.  A common misconception is automation on the web is unstable, high maintenance, and hard to scale because websites simply change too often — resulting in a constant cat-and-mouse game.
However, after looking at over 1M automations, over multiple months, on Weeldi the numbers tell a different story. 45% of automations fail because the customer input data is wrong. These are issues like invalid log-in credentials, invalid account numbers, incorrect scheduling, incorrect payment information, and attempting to capture data that doesn't exist. Weeldi’s API and User Interface allows users to update this input data on demand and classifies errors by customizable error type with supporting screenshots, so users can quickly (and automatically) identify and fix automations failing due to incorrect input data. Ultimately, if you want automation to succeed you need your input data to be clean and up-to-date. 43% of automations fail because of multiple website versions. This is most commonly believed by users to be a change with a website, however it is instead a new version of the same website, which they have yet to come across while automating. As you scale the Job volume behind each automation you see different scenarios you didn’t see before. This is commonly a result of regional differences, AB testing, or different account types on the same website. Weeldi’s automation engine supports multiple flows per website allowing customers to record multiple automations per website. Then Weeldi uses AI to figure out which automation works per Job and memorizes this for future runs, increasing automation success and removing painful guesswork for users. Simply put, volume strengthens automation. 7% of initial automations fail because of website performance issues. This means the website is down, loading too slowly, not returning a query. Generally, these are temporary issues and occur during website updates or more often on websites pulling data from several legacy systems, as at any given moment one of those systems may be down. (e.g. vendor websites like utilities, telcos, suppliers, etc.) Weeldi enables reattempt scheduling by automation, which means if a website is down, slow, or not returning an expected result, Weeldi will reattempt within a configurable timeframe until the website can complete the automation successfully. 4% of automations (if using RPA) will fail because of changing elements. Websites are built using HTML tags. A common example, a pay bill button may be implemented as element type BUTTON, but upon a new website update, it may have been changed to DIV, SPAN, or INPUT. These are tricky because while nothing may change to the human eye, these tag changes will stop traditional automation tools like RPA dead in their tracks. Since Weeldi automatically solves for these changes (as described below) instead of tracking failures, we tracked how often our automation engine had to solve for these changes. Weeldi’s proprietary approach to automating on the web takes into account a more complete description of the webpage (not just HTML tags) and uses AI to add an abstraction layer above the webpage, allowing the Weeldi automation engine to view web pages like a human. This means if an HTML tag changes, an element moves, or an element is named differently, chances are very high Weeldi will navigate through it ensuring the automation completes successfully. .67% of automations fail because of wholesale website UI changes. Wholesale website UI changes are often assumed to be the primary reason for web automations breaking, however wholesale website UI changes are by far the least common reason why Weeldi automations break. The reality is wholesale UI changes take a lot of time and endanger the user experience, which means they don't happen frequently. Weeldi’s automation engine is equipped with tools that make it clear when automations are failing and why they are failing, including supporting screenshots and error details. This makes it immediately obvious when a complete website UI change has taken place. The next step is adjusting to the new change and this is where Weeldi's No Code Recording Engine (a Chrome Extension that runs in your Google Chrome Browser) allows non-technical users to quickly record a new or updated web automation in just minutes. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required.
How an up and coming machine learning technology overcame every barrier to automatically collect subscription and service data from vendor web portals and SaaS applications
MISO3’s line of business requires getting data in hard-to-reach places An intelligent subscription and service tracking company serving the needs of enterprise-level companies, MISO3 gives its clients insight into its vendor subscriptions and usage. Think about it: vendors like Zoom, Microsoft 365, Salesforce, Amazon AWS, CenturyLink, Bloomberg, and the list goes on (and on and on) — there are few businesses left that are not reliant on licenses, SaaS, PaaS, and/or IaaS subscriptions to power their business. The data required to manage these vendors spans inventories, portals, and invoices, making visibility as well as the ability to make data-driven decisions almost impossible. For most clients, MISO3 was able to deliver on its promise rapidly and with few hurdles. And then… one day… MISO3 came up against a challenge, and a big one. A Fortune 100 client and one of the best-known financial and credit card companies in the world came to MISO3 with the challenge of understanding and optimizing subscription costs for their financial data and intelligence provider. MISO3 knew exactly what it needed from the provider to better understand the client’s spend and utilization of their subscriptions and services. However, with no inventory-targeted APIs available and data scattered throughout a clunky provider portal, they needed a predictable and scalable way to get control of this multi-million-dollar expenditure. They turned to Weeldi. Headquartered in the San Francisco Bay Area, Weeldi is the easiest way to automate any process on the web with no coding required. A common use case for Weeldi is filling gaps (where vendor portals and/or SaaS applications lack API functionality) by enabling customers to automate tasks directly through the vendor portal or SaaS application user interface, common tasks include extracting data, executing transactions, and paying bills. For MISO3, Weeldi’s web automation platform would be the cornerstone of meeting its client’s need to get its vendor data out of multiple locations quickly, accurately, and with as little friction as possible. MISO3 prides itself on being the first “easy-button” in vendor inventory discovery, extraction, and alignment. What once took businesses months of labor-intensive research and data cleaning, can now be done in only hours; minimizing user-prone errors and discovering hidden costs that often uncover waste and security vulnerabilities. (The alternatives were hours of manual labor every month or attempting to engineer a homegrown web automation solution with all the costs, delays, and opportunities for error that creates — rarely an efficient path to client satisfaction.) Data: it’s everywhere, but still so hard to find A little background on the problem faced by MISO3’s client (and, really, all of its clients, and almost every business everywhere). It’s all about “the other guys”: the SaaS companies, the IaaS companies, and the PaaS companies —and the mixed blessing they provide. In 2020, the overall spend on SaaS products was up by 50% compared to only two years before. (Data has yet to be collated, but we’d bet it’s going up again in 2021.) The average company uses 137 different SaaS apps, with each employee using at least 8 different ones. (Thanks to Blissfully for this data.) With subscriptions varying from person to person and from department to department across some or all of those 137 vendors, most companies are tilting at windmills when it comes to effectively managing subscriptions. They’re oversubscribed, leaving ex-employees licensed (sometimes, to do ill), and generally overbuying across the board. They do have access to data to see what’s what, of course. But — because they need to manually chase and capture it — the data is rarely in real-time… or complete… or accurate. It makes a difference in the same way Monday-morning quarterbacking makes a difference. This is where MISO3’s M3 platform powered by data extracted from Weeldi helps. Overcoming logins and CAPTCHAs and other roadblocks—automatically Weeldi was brought in by MISO3 to solve these problems and more. What Weeldi does that others can’t is to create stable automations on vendor portals and SaaS applications that neither Weeldi nor its clients control. Weeldi ’s platform allows customers to create automations on the web when APIs simply aren’t enough, or when APIs simply aren’t available (e.g., for billing data and subscription info — the exact kind of data MISO3 collects for its clients). Weeldi has a menu of available vendor portal automations and provides the flexibility of “built to order” automations that customers can create in minutes with no coding required. Automations can uncover and capture the data or execute transactions buried in vendor portals (multiple pages, behind a login, multi-factor authentication, CAPTCHAs, requiring scheduling or filtering, on the user interface, in emails, on unstable websites, etc.) or within SaaS applications. Weeldi’s flexible and reliable automations for the web were the ideal fit for a client like MISO3, a company with a business model reliant on retrieving information from vendor portals and SaaS applications, and were central in MISO3 sustainably supporting the business of one of the best-known credit card and financial companies in the world. A solution delivered in days; results that last a lifetime Within days of receiving specifications from MISO3, Weeldi delivered complex automations that extract — from behind logins (and tricky multi-factor authentication) — critical data, some requiring filters by date and account number, some from the user interface, and even some embedded in emails. Files are then automatically renamed, and folders are created in MISO3’s sFTP to feed their ETL (extract, transform, load) process back into the M3 platform. This was not a “one and done” for MISO3, either. The automations created by Weeldi give it the ability to quickly create connectors to any vendor portal or SaaS application. Additionally, Weeldi provides precision, actionable visibility to MISO3 (and all Weeldi clients) on how their data is flowing. Weeldi can say exactly where and why the data isn’t flowing and provide direction (along with screenshots) and tools to fix it. In the end, MISO3 had a solution that served its immediate need with its client, and a solution it could use repeatedly as it continues to build its business. “Before you can make data-driven decisions for your organization, you must first know what you have,” says David Baule, CEO of MISO3. “Gathering this data manually is both cumbersome and time-consuming. Weeldi gives us the power to quickly scale to any web-based vendor portal or SaaS application at a fraction of the time of doing this manually. The result, quick and frictionless ingestion of customers’ data so M3 Users can make informed decisions on all of their vendor commitments.” Weeldi transformed disparate, dynamic, and sometimes unstable Vendor Portals into stable, scalable APIs, providing MISO3 with the detailed data required to analyze and optimize their customers' subscription costs. All delivered with no coding required by MISO3. A solution delivered in less than two weeks that will last a lifetime. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required.
EXECUTIVE SUMMARY:
Motus is the definitive leader in solutions for businesses with mobile-enabled workforces helping over 2,300 companies and 300,000 mobile workers simplify reimbursement and management of mileage, mobile devices and remote work. After a recent acquisition, Motus faced the challenge of automating its acquisition’s data collection processes from over 20 vendor websites, spread across 13 countries and in 10 languages. Motus had a simple objective: quickly create automated processes to identify, capture, and consolidate the critical business data it needed from anywhere on the web it was living. So Motus explored every opportunity to automate data collection on the web. After a pilot with a well known RPA (robotic process automation) platform, and a failed experiment with an open source tool, Motus chose Weeldi for a pilot project. When Weeldi delivered five stable automations within its ten-day risk free pilot, Motus chose Weeldi as a partner. The company currently enjoys more than 90 automations in 10 different languages, collecting endless amounts of critical usage and spend data for their customers. The problem: complex automation of manual processes on the web to better handle scale The nature of the mobile workforce is often chaotic. Different devices used by different staff; working in different places and timezones; different needs for tech, transportation, and just about everything. As the company that simplifies management for all that (and more) for over 2,300 customers, Motus has to think and act at scale. In this case, it meant being able to consistently and completely collect critical data for its clients from multiple vendor websites regardless of the hurdles at each step (CAPTCHAs, Two-Factor Authentication, instability, different website frameworks, etc.). Prior to acquisition, the acquired company’s processes could be handled with the right people, manually. But after being acquired and integrated into Motus, the need for scale increased exponentially, and manually combing through thousands of webpages a month was no longer an option: automated processes were needed to prepare for the increase in volume, while the manual labor was directed to higher value tasks. The search for a solution, part 1: start, sputter, stop After working with a well-known RPA (robotic process automation) company to create a proof-of-concept for a single automation, Motus turned to Selenium, an open-source browser automation tool (primarily used for app testing) to build it. After a development process that took the better part of two months, one automation was built. However, it proved to be a house of cards, unstable and unable to accomplish the single task it was developed to execute on the web. So Motus looked elsewhere and decided on a 10-day pilot with Weeldi, a San Francisco-Bay Area company that enables companies of any size or technical ability to stably automate processes on the web, at scale, with no coding required. The solution, part 2: automation at a record-setting pace, and we’re just getting started In the 10-day pilot, Weeldi delivered five full-featured automations to help Motus automate complex manual backend processes on vendor websites, and another 40 processes just days after the official start of the engagement. Weeldi was able to create stable automations on the web for Motus (without requiring any coding work from the client), solving challenges such as login, OTP resolution, task scheduling/completion, and more. Motus now enjoys more than 90 automations, in 13 countries, and 10 languages. Other “Weeldi wins” for Motus include:
With Weeldi, Motus enjoys the freedom to quickly create dependable automations for endless tasks across the enterprise and can better and more completely utilize the skill sets of its staff. Perhaps most importantly, because of Weeldi’s flexibility, Motus essentially has every automation on the web it will ever need at its fingertips. About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required.
8 areas where you can leverage AI today for automation on the web.
Artificial Intelligence (AI) is a hot and highly marketed topic, but the application of AI for automation on the web is still in its infancy. This means those laying the right foundation today will be rewarded with the compounding value of less friction and more success with future automations. Here are 8 areas where you can begin leveraging AI for automation on the web. 1) Login Many automations on the web start by getting through a login page. AI can be leveraged to simplify the login step by looking up standard login components such as username, password, and “remember me” functionality, minimizing obstacles like MFA (Multi-Factor Authentication) and CAPTCHAs, and increasing the probability of automation success when there are login page changes. 2) Flow selection A growing number of websites have multiple versions based on region, customer type, M&A (Mergers & Acquisitions), and/or A/B testing. Using AI you can simplify the support of multiple website versions. This can be achieved by recording multiple automation flows, by website version, then using AI to select and navigate to the correct flow. This approach adds scale and stability to the automation by eliminating the need for a human to micro-manage variability of website versions. 3) Managing website issues Traditionally, website instability is a key culprit for breaking automations on the web. AI can help solve the most common instability issues like slow-loading webpages and webpage timeouts. It can do this by intelligently adjusting bot wait times by website, proceeding when critical components of a webpage are available (but noncritical components are not) and automatically reloading webpages with timeouts. 4) Formatting Bot automation flows are guided by data such as dates, account numbers, and SKUs. However, the way this data is presented on websites may vary from how you store it in your database, and even if you harmonize the two, eventual changes to the presentation of data on the web will break your automations. AI can help solve this problem by recognizing data patterns when websites make changes to the way they present data. Common examples are changes in date formats (i.e. 01/17/2021 vs. 17/01/2021), account numbers (i.e. 001234567 vs. 1234567), and SKUs (i.e. TESLA-S vs. Tesla Model S). 5) Nomenclature Change on the web is constant as websites evolve to meet their users' expectations. For example, you may download your utility bill each month by clicking on a link that states “Invoice”, but next month that same link may state “Bill”. These types of seemingly obvious changes to the human eye will break traditional bot automations. By using AI you can better recognize relevant nomenclature changes such as synonyms and increase your probability of success automating on the web. 6) Navigation Website changes go beyond nomenclature changes. Sometimes nomenclature stays the same, but the elements in the User Interface change. A common example is a website modernizing clickable links by transitioning them into buttons. This kind of change will break traditional bot automations. By storing the goal/intent of your automation rather than just the HTML element location and raw browser interactions you can leverage AI to successfully navigate website changes. 7) Pop-ups Websites use pop-ups to gain the attention of their users. Unfortunately, this can break brittle, bot automations. By using AI you can intelligently recognize pop-ups, the content within them and derive whether your automation should fail or close (or acknowledge) the pop-up and proceed with the automation. 8) Scheduling Some Bot automations run on a preset schedule based on an understanding the automation can be executed successfully at that time. But, what if your scheduled date is wrong. A common example is you command your bot automation to download a report on a specific date, but continually the report isn’t available until 2-3 days later. AI can help you solve this problem by learning the optimal schedule based on historical automation success and suggest the appropriate schedule adjustment. Why Weeldi? As companies attempt to automate predictably on the web they must consider the web's dynamic nature and look to pattern recognition technologies like AI to help manage this variability. Companies like Weeldi have started laying the groundwork by designing automations that focus on user intent, rather than exclusively on static navigation steps, and then using AI to successfully navigate to the users expected outcome. This approach proves to be a sturdy alternative for automation on the web where change is constant, broad, and unpredictable.
1) Encryption Sensitive data like passwords and PII (Personally Identifiable Information) required to run automations should be encrypted in transit and at rest with a cloud-based automation vendor. Also, sensitive data like passwords should never be displayed in clear through the application UI. 2) Transparency Access logs should be made easily available to you showing every time-sensitive data is accessed, including username, time of access, and the data accessed. Neither your users nor a cloud-based automation vendor should not have the ability to edit or remove these access logs. 3) 2FA (2-Factor Authentication) 2FA adds another layer of time-sensitive, security to the login process, reducing the chances of your account being hacked. This layer can be an OTP sent to your mobile device or an auto-generated code via a 2FA app. However, it should be noted, as hackers get more sophisticated in their ability to hack into SMS via flaws in SS7 protocols 2FA apps are preferred for optimal security. Cloud-based automation vendors should provide the flexibility to require 2FA at the organization level or individual user level. Commonly used 2FA apps are Twilio Authy or Google Authenticator. 4) KMS (Key Management Service) Integration KMS Integration allows you to integrate with your current KMS system to create and control the encryption keys used to encrypt your sensitive data. It also enables transparency into who is accessing your sensitive data and provides you with full and immediate control over who can continue to access your sensitive data. Commonly used KMS tools include Amazon KMS and HyTrust. 5) Secret Manager Integration Secret Manager integration gives you the ability to use your Secret Manager tool of choice to store sensitive data like passwords, control access to sensitive data, maintain transparency into when sensitive data is accessed and centrally manage sensitive data. Commonly used Secret Manager Tools include Amazon Secrets Manager and Google Secret Manager. 6) SSO (Single Sign-On) SSO integration allows your users to log into a cloud-based automation provider with a single ID and password following the SAML 2.0 protocol, usually centrally managed through an Identity Management provider. Commonly used Identity Management Providers include OKTA and OneLogin. Why Weeldi? As companies automate more tasks requiring sensitive data like passwords or PII (Personally Identifiable Information) they are faced with the challenge of ensuring their automations run securely. Cloud-based automation solutions like Weeldi provide transparency into sensitive data access, best in class security using tools like encryption, 2FA (2-Factor Authentication) and SSO (Single Sign-On), as well as integration with 3rd party KMS (Key Management Services) and Secret Manager tools to ensure your sensitive data is secure and in your control. Weeldi and Mobile Solutions (now Brightfin) partner to further automate mobility management9/25/2020
Mobile Solutions (now Brightfin) Partners with Weeldi to Enhance Managed Mobility Software Automations
San Francisco, CA and Centennial, CO – September 25 2020: Weeldi, a San Francisco Bay Area-based company innovating the way processes are automated on the web, and, a leader in Managed Mobility Services, delivering expense optimization, unified endpoint security, device management, and best in class support, have partnered to add additional automations to MAX, Mobile Solutions (now Brightfin) proprietary mobility management portal. Automations can be difficult due to the dynamic nature of the web. Partnering with Weeldi has enabled Mobile Solutions (now Brightfin) to easily and quickly add new automations, while also providing the flexibility to manage and run automations through Weeldi’s web service API and UI. With Weeldi, Mobile Solutions (now Brightfin) is automating about 50,000 tasks critical to managing their customers’ expenses, inventory and usage monthly. MAX, Mobile Solutions (now Brightfin) innovative mobility management solution, was designed from the ground up built with the user experience in mind offering in-cycle analytics and delivering a unified platform for all mobility management needs. Weeldi’s API strengthens Mobile Solutions (now Brightfin) scale and frequency of automations, running a high volume of daily automations in a condensed timeframe of only a few hours. Weeldi also integrates with Mobile Solutions (now Brightfin) Amazon Key Management Services for encryption of their customer credentials. “Mobile Solutions (now Brightfin) is committed to using innovation as a vehicle to scale and deliver the best possible experience for their customers. This pushes us and makes the opportunities in our partnership almost limitless. We’re excited about our partnership.” said Moe Arnaiz, Co-Founder and CEO of Weeldi. Jim Johnson, Board Member of Mobile Solutions (now Brightfin), said, “We couldn’t be happier with our partnership and experience with Weeldi. From the fast, low risk deployment to running over 1,500 daily automations, Weeldi elevates our ability to automate tasks, providing up to date and timely data for our customers.” About Mobile Solutions (now Brightfin) Mobile Solutions (now Brightfin) is a SaaS company that provides a complete managed mobility practice. We make it easy for our clients to optimize IT spend, reduce data consumption, secure mobile endpoints, and simplify the management of their entire fleet of devices, all with best in class support. For more information, visit https: www.brightfin.com/ About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required. To learn more, visit https://www.weeldi.com/
5 approaches to automation that are too rigid for the web.
Miles Davis could pick up his trumpet, step into a quartet, listen, adapt and produce beautiful Jazz that worked. Similar to a great Jazz musician, a good automation approach for the web has the flexibility to process dynamic inputs, adapt in near real-time and deliver just the right output. However, most traditional automation approaches are rigidly designed with little ability to adapt to change, making them brittle when deployed on the web. Below are 5 approaches to automation that are too rigid for the web. 1) Using X-Y coordinates to locate objects Similar to longitude and latitude on a map, this approach relies on the objects on the screen to be predictably in the same location time and again, but the web is innately dynamic and objects move or could be presented differently based on the browser, device type and resolution. Advertisements, message notifications, report filter options and UI design modifications are examples of changes that will break any automation that relies on X-Y coordinates. 2) Extracting the DOM structure and looking for elements by name or structure The DOM is a tree structure of all the elements that make up a web page. Traditional automations extract the DOM structure and look for elements precisely by their id, class or position. This approach is prone to instability as many websites eschew naming DOM elements entirely, arbitrarily change ids and class names or duplicate ids in violation of DOM specifications, causing automations to break. Also, web development trends are moving toward websites that load dynamically as you scroll through the webpage, defer loading portions of a web page, or dynamically build portions of the page after it is loaded, to provide a more seamless experience to customers with lower bandwidth. While this improves the user experience, it confuses and breaks automations that depend on a static, predictable DOM structure. Finally, most DOM violations aren’t a stumbling block for the user experience because web browsers like Chrome and Firefox do a fantastic job of masking poor adherence to web standards. But, these DOM violations will break automations and require arduous collaboration from operations and software development to investigate and resolve. 3) Javascript containers (e.g. React, Angular, etc.) Javascript frameworks help software developers build consistent user experiences, however, they also hijack user interactions like mouse clicks and scrolling, which can render automations powerless, causing them to fail. 4) Using shortcuts to bypass the User Experience To increase the speed of automations, some software developers decide to bypass the website user experience by taking shortcuts to complete automations faster. Taking these shortcuts will inevitably lead to issues like slowing down (or crashing) your target website, IP Blocking, getting auto-logged out and/or accessing incomplete data. Short-cuts appear to be great until they don’t work and they then become very difficult to investigate and repair as they were not based on user experience to begin with. 5) Using web testing automation tools (e.g. Selenium, etc.) To accelerate automation, rather than start from scratch, software developers may choose to utilize web testing automation tools like Selenium. However, these tools have a propensity to get jammed-up by rogue javascript libraries, such as the ones often embedded in website analytics tools. So when you’re automating on the web you must ditch the traditional rigid approach for an adaptive modern approach that produces beautiful automations that work. Why Weeldi? As companies attempt to automate stably on the web they must consider its dynamic nature. Weeldi’s modern approach to automation is flexible and adaptive, enabling companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required.
6 + (1) tips for deploying your automations in the cloud at scale.
Automation improves how your business scales. But, how do you ensure your automations can scale? 1) Scale-Up Your automations may vary by volume or include tight time constraints. To ensure on-time, successful delivery of automations, you must be able to predict server capacity for peak usage and have the flexibility to deploy additional server capacity in real-time to guarantee timely execution. 2) Scheduling You must integrate the scheduling and prioritization components of your automation engine with your servers to provision adequate capacity to complete your automations on time. The scheduling and prioritization components to consider include: urgency, probability of success, concurrent login acceptance (if automating behind logins), the number of reattempts required for automations to complete successfully, amongst several others. 3) Scale-Down / Reduce Costs To run your automations economically, you must have the ability to reallocate server resources in real-time for automations that require them, or turn down servers during periods of lower automation activity. 4) Configuration Flexibility Not all websites are optimized for all browsing environments. One website may be friendlier on Chrome and another on Firefox. Further complicating matters, the process you’re trying to automate (i.e. downloading a file vs. submitting a web form) may have different probabilities of success depending on the configuration. By having multiple browsing configurations deployed in the cloud you can systematically A+B test your automations to identify configurations with the highest probability of success and use those configurations moving forward. Common configuration variables include: using Proxy IPs, using older browser versions, turning off 3rd party tracking amongst several others. 5) Security To protect your data, it's important to disconnect data storage from the actual processing of the automation. This enables you to secure the automation independently of the storage of the automation result, ensuring that anything bad your automation might pick up on the web does not contaminate your stored data. 6) Monitoring On the web, many items can degrade or even crash your automations' performance. To run stable and timely automations, it a requirement to have real-time visibility and monitoring of server health and usage 6*) Target Website Acceptance To ensure your automation is accepted by the target website, you must have IP flexibility including provisioning dedicated IPs, untainted IPs, a range of customer IPs or in some instances leveraging Proxy IPs. Why Weeldi? As companies increase the volume of tasks they automate, they are faced with the infrastructure challenges of ensuring their automations run successfully, on time and at scale. Cloud-based automation solutions like Weeldi provide out-of-the-box functionality that enables you to automatically scale to meet your workload. Weeldi empowers market leader, vCom Solutions “to automate processes, where RPA falls short”3/18/2020
Web data extraction solution eliminates the need for relying on APIs or using costly RPA tools to cleanly extract data from the web.
SAN FRANCISCO, California — March 18, 2020: Weeldi, a San Francisco Bay Area-based company innovating the way processes are automated on the web, has partnered with vCom, the premier IT spend management firm for the mid-market, providing vCom with a fast path to automate processes for data extraction on the web enabling them to immediately capture critical business data from thousands of web pages at scale via the Weeldi API. Cofounded in 2019 by Moe Arnaiz and Mathieu Guilmineau whose last SaaS company was acquired by a $6.7B private equity firm — Weeldi provides customers a fast and reliable means of automating processes on the web, filling gaps where API’s are unavailable or incomplete and tools like Robotic Process Automation (RPA) are difficult to deploy and costly to support. “RPA is top-of-mind for businesses as they look to automate repetitive tasks and while it can work well in a stable and static private server environment, automating processes using RPA tools quickly gets unwieldy in the wilds of the web where you run into common challenges like OTP, Captchas, IP Blocking, multiple website versions, multiple web frameworks, scheduling, website timeouts, and the list goes on. Trying to solve these problems with RPA leads to an endless, expensive cycle of customization and maintenance, as companies struggle to reliably automate processes on the web” said Arnaiz. “With Weeldi, we’re offering an out-of-the-box solution, deliberately leveraging technologies like AI and modern web drivers to solve the challenges of automating processes on the web all in one platform, to make automation of processes on the web more stable and predictable from project and cost standpoint,” said Guilmineau. “I couldn’t be happier with Weeldi,” says vCom COO Sameer Hilal. “All the data we need from invoices, reports, and portals are being collected automatically. We’re getting exactly what we need, faster and cleaner, and freeing up our development team for other projects. And the best part is we’re only paying for successful outcomes: we’re not paying for development hours or RPA bot hours: we’re only paying for the extracted data.” A SaaS tool, Weeldi has simplified data extraction for vCom across more than 30 telco and technology vendor websites, extracting text and capturing data from web pages and transforming it into an easily consumable API, enabling integration at scale into vCom’s award-winning vManager software. Embedded in Weeldi’s platform are solutions to solve a range of challenges synonymous with the web. (Read more about Weeldi vs RPA) “vCom is using the Weeldi API to run more than a thousand automations on various schedules from monthly to on-demand, and able to build new automation instructions in minutes,” said Hilal. “We deploy RPA to automate other internal processes, but for this use case, on the web, there’s no comparison.” About vCom Solutions vCom is a cloud-based software and managed services company focused on helping enterprises manage IT spend from procure-to-pay. vCom improves visibility and control within a single software while decreasing expenses for networks, mobile, cloud, collaboration, and SaaS technologies. To learn more, visit http://vcomsolutions.com About Weeldi Headquartered in the San Francisco Bay Area Weeldi enables companies of any size or technical ability to stably automate processes on the web, at scale, through its web service API or user interface — with no coding required. To learn more, visit https://www.weeldi.com/
10 obstacles you need to consider when using Robotic Process Automation (RPA) bots for unattended automations on the public internet.
RPA on the public web RPA bots are powerful tools, however if you are attempting to use RPA bots for unattended automations on the public web you’ll need to ensure you have solutions for the following 10 obstacles. 1. Log-Ins The processes you want to automate may be on the Deep Web (behind login credentials). To automate processes behind logins you’ll need to make sure your RPA bots can securely access the credentials necessary to complete your unattended automations. 2. Two-Factor Authentication Websites may require two-factor authentication when logging in from a new or unrecognized web browser. In addition, this step can be triggered at the discretion of a website at any time. You’ll need a solution to avoid and resolve two-factor authentication. 3. Captchas Websites may require you to solve captchas before granting access. You’ll need a solution to solve or avoid captchas. 4. IP Blacklisting Websites may blacklist critical IP addresses if you are accessing them too fast, too frequently or concurrently. You’ll need a solution to ensure you do not trigger IP blacklisting. 5. Scheduling You’ll need a flexible solution to schedule frequency of RPA bot attempts and reattempts (for failed) unattended automations. 6. Schedule-and-Return Some scenarios will require you to first initiate your automation and then return within a defined time frame to complete it. You’ll need a flexible solution to manage schedule-and- return automations. 7. A/B Testing Websites can have multiple versions of their site depending on region, customer type and/ or A/B testing. To adequately support this common scenario you will have to build and support multiple versions of your bot. 8. Intermittent Downtime, Time-Outs and Latency Websites on the public internet can be unstable so your RPA bots will need to be equipped to manage instability to ensure you can complete your unattended automations successfully. This means not aborting unattended automations too early and automatically reattempting others at a later time. In some instances, it may take several attempts over multiple hours before an automation completes successfully on a public website. You’ll need a solution to manage this website instability carefully. 9. New Front End Frameworks and Elements As the web evolves so do the frontend frameworks and elements. Your RPA bots should be able run in frameworks such as AngularJS and ReactJS and elements such as modals and hidden fields. In addition, your RPA bots should evolve to account for new frameworks and web elements as they are introduced. In the world of RPA bots this may mean building a new bot. 10. Rogue Scripts, Insecure Connections and Mixed Content Some websites include rogue scripts, which will trigger timeouts and others insecure connections or content, which result in modern browsers rejecting all or part of the page. You’ll need a solution to manage through these obstacles. Why Weeldi While RPA bots can be powerful tools, they often require costly, intricate configuration and even custom development to deliver an enterprise-ready solution for automating process on the public internet. Alternatively, considering purpose-built solutions for automating process on the public internet could save you time and money. As an example, companies like Weeldi provide solutions to these 10 obstacles and many others, out-of-the-box, with no costly configuration or coding required. In addition, you pay by the results, not per bot hour or the individual bot.
How businesses are moving from risky home grown “scraping” solutions into secure, comprehensive Web Data Integration (WDI)
The current state of web data Web data is the boundless unstructured data living on websites, web portals and SaaS applications–simply put, it’s the data in your web browser. For years, web data has been something you can see, but not really “touch” or incorporate into your business systems without accessing each website via an API or building homegrown scraping solutions. But that’s about to change. Finding the fuel to move forward Often, the data a business needs to fuel success can be accessed by APIs, but not always, as there are countless non-API sources of information on websites, web portals, and even some SaaS applications. Take for example many Mobile Telecom Companies who offer APIs to customers to help manage expenses, but leave out critical pieces of data, like current international usage, which is invaluable for avoiding costly roaming charges. This usage data may not be in the API, but is instead sitting as unstructured web data, trapped in a customer web portal...ripe for consumption if it could only be easily accessed at scale. Moving past relying (solely) on APIs While APIs are a standard means of data transfer, that doesn’t necessarily make them the most expedient or ideal. Some API access processes can take weeks, months, or longer to approve and set-up, and these delays raise risks of failed deployments, missed revenue opportunities, and poor customer service. But assume API setup goes smoothly: what if your API vendor sunsets support, or decides to make significant changes that make it more difficult to work with their API? The ability to independently access web data moves from being a “nice to have” to a must, as does the need for a modern Web Data Integration (WDI) tool that can extract web data across multiple websites with the same scalability, reliability and structure of an API. According to 2019 research from Opimas, spending on web data extraction was $2.5B in 2017 and predicted to reach nearly $7B by 2020, with the majority of spend on internal homegrown solutions. But while the overall web data extraction spend is trending up, Opimas expects a significant transition from internal spending (with expected growth at 30%) to external spending (with expected growth at a whopping 70%). (1) So why the move toward more external solutions? In their current state, internal solutions for web data extraction are often unstable and send valuable software development resources down a proverbial rabbit hole of custom coding and neverending maintenance in a attempt to manage: data extraction templates for every data source, IP blacklisting, CAPTCHA, 2 factor authentication, pacing, data access across multiple pages / websites or data behind logins, file downloads and scheduling — to name just a few. Signs are pointing toward use of modern Web Data Integration (WDI) solutions that quickly transform any unstructured web data into structured APIs — with no coding required. Specifically, faster, more reliable, more scalable, and more affordable alternatives to legacy homegrown and outsourced web data extraction solutions. How modern Web Data Integration (WDI) differs from “scraping” Modern Web Data Integration (WDI) tools are built to deliver scalable, reliable web data extraction out of the box without requiring software development for either configuration or maintenance. Whereas scraping is a disjointed process of collecting web data that requires software developers to build and reactively maintain custom scraping routines, typically built with a hodgepodge of open source tools. A modern Web Data Integration (WDI) tool gives you “out of the box” ability to:
Scraping requires hourly, salary and/or contract specialist in product management, software development and QA to build and maintain the same functionality that is available out of the box with Web Data Integration (WDI) tools as listed above. Plus you to take on the risk of:
To DIY, or Not DIY? SMB and enterprise adoption of modern Web Data Integration (WDI) tools is on the rise. While there are many legacy, home-grown, outsourced and on-premise solutions still deployed to solve the “web data extraction needs” of competitive businesses, most do not deliver the modern ease of use, reliability and scalability companies need to make web data an integral part of their business. Consider this: just as Stripe has done for online payments and Twilio for business communication, modern Web Data Integration (WDI) tools are now doing to empower organizations in all markets and of all sizes to capture and act on the endless ocean of web data. REFERENCES 1: A. Griem, & O. Marenzi, Web Data Integration — Leveraging the Ultimate Dataset [White paper], retrieved March 2019, Opimas Research |
WeeldiTransform a website into an API in seconds w/ no coding required. 
Archives
September 2021
Categories |
